Infrastructure Renaissance

Infrastructure is Code! FTW!

Wait… WTF does that even mean?

On the one hand this is a rallying cry for those of us in the configuration automation space, on the other hand it confuses innocent bystanders.

@zhenjl: @botchagalupe but ops ppl have always coded using perl, tcl (if u remembr that :)…now we just have ‘sexier’ langs like ruby and python (original)

Sure, there has always been scripts, which is technically code, but this is missing the point a bit.

For starters, getting ops teams to recognize and steal all applicable processes and tools from developers to produce and version there scripts would be a win, but that’s just the starting line…

The point isn’t just that you can write some scripts to help you do something with your infrastructure then exit.

The ‘Infrastructure’ IS an application, a long running process with inputs, outputs and state. Or at least infrastructure is very close to becoming one…

Everyone paying attention knows something is happening… what is it?  Hazy, foggy, smoky, cloudy, whatever…

EC2 is the herald of an infrastructure enlightenment, leading us out of the dark ages.

(We’ll ignore the history of virtualization technology, and ESX Server for now, but I’m sure someone has an opinion on all this.)

The bottom line is there are resources, and I can manipulate them through an API. With EC2, these are compute resources of various shapes and sizes, with some storage resources.

Virtualized networking is not exposed through a public API yet, but we’re still climbing abstraction mountain together and we all know it is just a little higher in this cloud. (What’s up Cisco, where you at? Oh, there you are… and these guys too… thanks Ben )

At some point in the not too distant future, web service calls and a credit card will be enough to create and manipulate arbitrary compute, storage and network resources.

So now what?

Until someone provides Sysadmin as a Service, there is still quite a bit of work left to be done. These resources need to be configured to provide services.

When adding capacity was a purchase order, then wait for weeks to rack and cable, taking a day or so to configure the machines was not rate limiting, and lost in the noise. When you can bring up new server instances in minutes, that same configuration step is conspicuous waste.

You might be thinking you’ll make images for each of your services and manage those. This approach to management exchanges the configuration drift of tradition IT management for image sprawl. Do those notes that got scribbled six months ago when the web server image was created have accurate details about how that machine is configured? Are you sure? The only way to know is start the VM and inspect it. Frankly, the same thing applies to the running instances started yesterday.

That’s where tools like Puppet comes in. The benefit is not just that a machine gets configured by running a script. Puppet is an application that audits your system configuration, and changes the details out of sync with policy. That policy is code with semantics that can be developed like software.

That’s the shift. Instead of thinking about managing server counts, we can reason about abstractions of service configurations. The pieces exposed by an API are not just instances of server and storage resources, but abstractions of services. The configuration of proxies, monitoring, middleware, web servers and databases are components to be composed in the dynamic ‘infrastructure’ application.

This is just the beginning, but I don’t want to spoil the rest of the story… at least not yet.

It is not necessary to change. Survival is not mandatory.

W. Edwards Deming