A Joining of the Tribes

Forgive him, for he believes that the customs of his tribe are the laws of nature!

-George Bernard Shaw

I’ve spent the last decade involved in software development on some level.

During that time, I’ve built things with Fortan, Java, C/C++, and recently Ruby at the core with Tcl, HTML, CSS, Javascript, XSLT, and SQL among other things as adornments. Never really did much Perl and as a result I suck pretty bad at regular expressions, especially by sysadmin standards (which will become more relevant in a minute).

Whachew looking at?

Can I get some WiFi around here?

I was a ‘Developer’. I had a tribe. Not a LISP druid or a kernel hermit, who tend to be solitary creatures, but a ‘yes, I can do that’, Linux-loving, little-bit-of-everything, do-it-for-the-team developer. (I used to think .Net stormtroopers were my natural enemy, but I’ve opened my heart and learned how to love. I digress. . .)

Parallel to the ‘Developer’ tribe in most organizations, often with a semi-antagonistic mutual dependence, there was always another tribe: ‘Sysadmin’. When Developers and Sysadmins got together, it sometimes felt like the dwarfs and high elves forced to work together by necessity. (I’ll let you workout which is which.)

But something is happening, something is changing. Part of the change has to do with clouds and what utility computing enables, part of the change has to do with the scale that successful web applications must account for and perhaps part of it is that it’s just about damn time. Sandwiched between all these forces, and perhaps in part as a catalyst, is a little open source project with big hopes and dreams: Puppet. (Or maybe I just think about Puppet too much)

All evolution in thought and conduct must first appear as heresy and misconduct
– G.B. Shaw (two for two)

Which brings us back to the tribes… One of the recurring themes at the Velocity conference on operations and web performance was that in this world of highly available, scaled and performant web applications, developers have to think about operations and admins have to think about the applications.

Throwing tools like Puppet into the mix (maybe with a little help from kickstart, fai, PXE booting or a request to EC2) where one can express infrastructure as code and the whole thing goes to another level.

A Reductive Labs client (who shall remain nameless, but has a kick ass Indian developer who was born in Canada, sucked at hockey and should probably give it all up to become a comedian) did an on-site Puppet training about a year ago, and Puppet has become such a part of the development culture that the developers are required to write the Puppet classes that will manage their applications. If there is an operational issue with Puppet managing the application, it is opened as a bug against that code. How cool is that?

Operations is development. It’s going to happen. . .

Some lines are being drawn, you can be a ‘Developer’ who understands not only the bits and algorithms but also the way those things need to behave across racks of servers (or virtual machines) or you can work on that little internal business rules engine. Or you can be a ‘Sysadmin’ who can think about what it takes to automagically scale an application to 100s or 1000s of servers, or you can change tape and plug in cable. Which tribe are you going to be in?

Still have I borne it with a patient shrug, for sufferance is the badge of all our tribe.

-William Shakespeare

And when we all get together to build something incredible and amazing, please don’t hold my feeble RegEx fu against me. (you can tease me a little, I deserve it)


9 responses to “A Joining of the Tribes

  • Nigel Kersten

    It’s interesting this came up at Velocity.

    At WWDC this year there were a few of us sysadmins who seemed to have all come to the same conclusion at around the same time, namely that sysadmins need to learn more from software development.

    I’ve been trying to apply the same mindset to large scale client management lately with a framework to run unit tests on candidate client images. Why don’t sysadmins use unit tests more? Why don’t we do peer review more?

    For me I think it was moving to Google that led to this epiphany, as this is literally the first company I’ve ever worked for whose primary business is software development, so we already have all these tools in place that are awesome for sysadmins, like Mondrian.

    An interesting point a friend made was that the benefits of peer review on all your scripts are indisputable, but if you simply don’t work somewhere where you have a group of peers who can review all your work (and I’ve been there), then the best thing you can do is get approval to blog about it and publish scripts externally.

    People like correcting mistakes in blog posts and suggesting improvements. If you put your work out in public, it’s not quite the same as immediate local peer review, but you can collect many of the same benefits, and that’s just from a purely selfish point of view, let alone the more altruistic aspects.

  • Bryan McLellan

    I’m a sysadmin. I usually use the IANAD line to avoid fixing people’s code when they think it’s a system problem but I have to prove it isn’t.

    My buddy at work, the lead developer came into my office one day and said that I am a developer, because I had more code checked in to the puppet repository (git) than most of the developers had checked into the product’s repository.

    So now there’s a little yellow piece of sticky note on my name badge outside my door that says “Puppet Developer”, taped over where it says “Systems Administrator”.

  • Puppet news links at btm.geek

    [...] Stories about people using puppet: Digg, Google, iLike. SA vs Devs. [...]

  • People Over Process » links for 2008-07-12

    [...] A Joining of the Tribes « fate = will && choice || circumstance “Operations is development. It’s going to happen…” (tags: via:im puppet operations developer/operations itmanagement) [...]

  • Dave Atkins

    I shared this with our tech team; great stuff. I’m definitely more of the ops tribe; only mildly interested in development theory. java, .Net, perl, php…whatever gets the job done. Bottom line is user exprience and whether things work reliably. But it is a constant balancing act–to solve the problems that are relevent and stay 1 step ahead (not 2 or 3) of requirements.

  • links for 2008-07-15 : Bob Plankers, The Lone Sysadmin

    [...] A Joining of the Tribes « fate = will && choice || circumstance “in this world of highly available, scaled and performant web applications, developers have to think about operations and admins have to think about the applications.” Been saying this for a while, that all good apps teams need a good sysadmin, too. >> Comments | ShareThis [...]

  • links for 2008-07-16 : Bob Plankers, The Lone Sysadmin

    [...] A Joining of the Tribes « fate = will && choice || circumstance “in this world of highly available, scaled and performant web applications, developers have to think about operations and admins have to think about the applications.” Been saying this for a while, that all good apps teams need a good sysadmin, too. [...]

  • Agile Infrastructure « fate = will && choice || circumstance

    [...] who has essentially crossed the Developer-Sysadmin divide (I’m not sure he is a chief of the new tribe, but he’s definitely a shaman), Luke became frustrated that he couldn’t write Puppet [...]

  • Developers and Sysadmins, Managers and Engineers: Unite | Heroix Blog

    [...] roles have been separated for far too long. Andrew Shafer blogged about the need for a “joining of the tribes” back in July 2008, and that post stuck with me ever since. I’ve been guilty of [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: