Infrastructure Renaissance

Infrastructure is Code! FTW!

Wait… WTF does that even mean?

On the one hand this is a rallying cry for those of us in the configuration automation space, on the other hand it confuses innocent bystanders.

@zhenjl: @botchagalupe but ops ppl have always coded using perl, tcl (if u remembr that :)…now we just have ‘sexier’ langs like ruby and python (original)

Sure, there has always been scripts, which is technically code, but this is missing the point a bit.

For starters, getting ops teams to recognize and steal all applicable processes and tools from developers to produce and version there scripts would be a win, but that’s just the starting line…

The point isn’t just that you can write some scripts to help you do something with your infrastructure then exit.

The ‘Infrastructure’ IS an application, a long running process with inputs, outputs and state. Or at least infrastructure is very close to becoming one…

Everyone paying attention knows something is happening… what is it?  Hazy, foggy, smoky, cloudy, whatever…

EC2 is the herald of an infrastructure enlightenment, leading us out of the dark ages.

(We’ll ignore the history of virtualization technology, and ESX Server for now, but I’m sure someone has an opinion on all this.)

The bottom line is there are resources, and I can manipulate them through an API. With EC2, these are compute resources of various shapes and sizes, with some storage resources.

Virtualized networking is not exposed through a public API yet, but we’re still climbing abstraction mountain together and we all know it is just a little higher in this cloud. (What’s up Cisco, where you at? Oh, there you are… and these guys too… thanks Ben )

At some point in the not too distant future, web service calls and a credit card will be enough to create and manipulate arbitrary compute, storage and network resources.

So now what?

Until someone provides Sysadmin as a Service, there is still quite a bit of work left to be done. These resources need to be configured to provide services.

When adding capacity was a purchase order, then wait for weeks to rack and cable, taking a day or so to configure the machines was not rate limiting, and lost in the noise. When you can bring up new server instances in minutes, that same configuration step is conspicuous waste.

You might be thinking you’ll make images for each of your services and manage those. This approach to management exchanges the configuration drift of tradition IT management for image sprawl. Do those notes that got scribbled six months ago when the web server image was created have accurate details about how that machine is configured? Are you sure? The only way to know is start the VM and inspect it. Frankly, the same thing applies to the running instances started yesterday.

That’s where tools like Puppet comes in. The benefit is not just that a machine gets configured by running a script. Puppet is an application that audits your system configuration, and changes the details out of sync with policy. That policy is code with semantics that can be developed like software.

That’s the shift. Instead of thinking about managing server counts, we can reason about abstractions of service configurations. The pieces exposed by an API are not just instances of server and storage resources, but abstractions of services. The configuration of proxies, monitoring, middleware, web servers and databases are components to be composed in the dynamic ‘infrastructure’ application.

This is just the beginning, but I don’t want to spoil the rest of the story… at least not yet.

It is not necessary to change. Survival is not mandatory.

W. Edwards Deming

About these ads

15 responses to “Infrastructure Renaissance

  • andy (thesethings) 's status on Sunday, 12-Jul-09 18:24:51 UTC - Identi.ca

    [...] Infrastructure Renaissance « fate = will && choice || circumstance [...]

  • BotchagalupeMarks for July 12th - 10:59 | IT Management and Cloud Blog

    [...] Infrastructure Renaissance « fate = will && choice || circumstance – On the one hand this is a rallying cry for those of us in the configuration automation space, on the other hand it confuses innocent bystanders. [...]

  • botchagalupe

    I believe it was Infrastructure as Code that was being discussed. I think you missed the point. IAC doesn’t mean servers it means creating the glue that makes infrastructure work (configuration). You can call it Infrastructure as an application if you like; however, I think IAC makes the point that as an industry we have ignored all the little pieces that glue all out servers together. These components have not been managed as an important business assets and as a result we don’t manage them like we would manage the source code of an application.

    As @zhenjl points out we have had perl scripts in the enterprise for years and he is correct. However, we do not treat those scripts as code the same way we do our precious applications. What products like Puppet and Chef are doing is making us arrange that information in a way that forces us to treat it as code and not just as scripts.

    John
    Johnmwillis.com

  • stochasticresonance

    botchagalupe,

    What am I going to do with you? I didn’t miss the point, I’m just two points ahead of you… ;)

    Infrastructure is code, infrastructure is data, infrastructure is an application, these are not mutually exclusive.

    Care less about the labels, and more about what it enables.

    We are moving towards enabling what we don’t have words to describe, so I expect some communication to be clumsy, but I also reject the idea that ‘infrastructure is code’ only applies to the ‘glue’. (especially since I say it all the time…)

    On another level, there is no ‘we’. There are organizations who have been version controlling their scripts and have sophisticated home grown configuration management tools specific for their infrastructure. At the same time, one doesn’t have to look too hard to find examples of application ‘code’ that isn’t being version controlled or treated like important business assets.

    Puppet doesn’t force you to version control anything. Puppet doesn’t force you to have dev, test and prod environments. Puppet just makes these practices seem like a really really good idea and makes the alternative seem irresponsible.

    @littleidea

  • botchagalupe

    All am saying is that somewhat like the term IaaS, Infrastructure as code is a good phrase to grow on. I think makes a very important point for the enterprise. The enterprise does not use version control their infrastructure “glue” (I will have a post on this tonight). They don’t manage their infrastructure the way they do their applications.

    I think the fact that you think you are two steps ahead of me .. lies the problem of our communication. I think we are saying the same you think we are not.

    John
    johnmwillis.com

  • stochasticresonance

    We are seeing the same thing.

    ‘Infrastructure is Code’ will be better when it makes sense to people like Jian Zhen.

    There’s another detail we’re leaving out, the technology is only half the equation.

    In many cases, the social engineering to fully leverage these tools and technologies is the enabler or deal breaker.

    This can get particularly sticky when SA’s have self identified and view ‘developers’ as the other. Version control is something ‘They’ use…

    Another post for another day…

  • People Over Process » Rackspace Cloud API & “Open Clouds” - Brief Note

    [...] API’s for clouds are important for the simple reason of making them easier and more flexible to develop on-top of. Instead of manually dealing with a UI (or “console”) you can programatically interact with the cloud to do your cloud operations. To pick one of the benefits of having a cloud API: developers hope to better automate the role of the sys admin, leaving developers more time to focus on the application rather than running the application. Much of the power that comes from the Amazon Cloud and others is based on using the APIs rather than requiring users to “manually” provision, “burst,” and otherwise administer their cloud-based applications. For more nuance on this, see one of Andrew Shafer’s recent posts on the topic. [...]

  • Agile Infrastructure with Andrew Shafer – Agile Executive 004 « The Agile Executive

    [...] which kind of boils down to the connection between Agile development and the IT department, esp. in trying to get IT to be Agile itself. Here are some, admittedly, poor notes from the [...]

  • People Over Process » What does Agile IT Management Look like?

    [...] which kind of boils down to the connection between Agile development and the IT department, esp. in trying to get IT to be Agile itself. Here are some, admittedly, poor notes from the [...]

  • turn the page « fate = will && choice || circumstance

    [...] of me wants to continue pushing the Infrastructure Renaissance forward in some capacity, and part of me wants to build something awesome, and part of me wants to [...]

  • John Arundel

    “You might be thinking you’ll make images for each of your services and manage those. This approach to management exchanges the configuration drift of tradition IT management for image sprawl.”

    I’ve explored further disadvantages of the imaging-only approach here:

    http://www.agilesysadmin.net/imaging-or-configuration-management

    and listed some ways to combine imaging techniques with automated configuration management via Puppet (or Chef).

  • The Rise of DevOps

    [...] is on the rise as a newly re-defined standalone discipline, as evidenced by increased number of good articles about it around the blogosphere. In this post, I am going to [...]

  • William Vambenepe — Steve Ballmer gets Cloud

    [...] Infrastructure Renaissance (doesn’t mention the word “devops”, but describes the phenomenon) [...]

  • Meatcloud Manifesto – The gauntlet is throw… « fate = will && choice || circumstance

    [...] Both conferences have an overlapping theme of the infrastructure renaissance. [...]

  • Meatcloud Manifesto – The gauntlet is thrown… « fate = will && choice || circumstance

    [...] Both conferences have an overlapping theme of the infrastructure renaissance. [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: