The End of the Infrastructure Engineer?

Is the age of the infrastructure engineer over? While I was at VMworld 2011, I listened to a speaker talk about how people like me—people who are conversant in virtualization, networking, storage, servers, and data center hardware—are going to have to evolve/migrate into the application development space in order to survive in the “new cloud world” (my phrase, not the speaker’s). This speaker isn’t the only one, either. This started me thinking. Are infrastructure engineers really a relic of the past, like punch cards?

I think we can all agree that the heavily-specialized infrastructure teams of the past—the networking team, the server team, the storage team—are no longer sufficient in the brave new world of converged infrastructure that blends networking, virtualization, and storage all together. I’d agree, in general terms, that IT infrastructure folks need to broaden beyond their core strengths into adjacent technologies in order to remain relevant. Storage engineers need to learn some networking and some virtualization. Virtualization engineers absolutely need to know storage, and networking is becoming a necessity as well. Networking engineers need to embrace virtualization and understand the impact of storage on their networks. Whether or not the silos ever come down fully or whether we’ll just “install sliding partitions,” as one colleague of mine said, isn’t yet clear. It is clear, though, that some blurring of the lines between these teams is a given.

Is it a given, though, that infrastructure engineers have to move up the stack into the application layer(s)? From an awareness perspective—meaning becoming more aware of the applications running on their infrastructure—I’d agree. Application development, though? That one I’m not so sure about. Yes, it will be/is helpful to understand what goes into application development and the infrastructure dependencies that are the result of the development choices. Again, that’s awareness, and yes—infrastructure engineers need enhanced awareness of adjacent technologies and the relationships with their core technology strengths. Awareness, though, is fundamentally different than being well-versed in application development.

Maybe I’m just being naive or ignorant, but regardless of how many layers of abstraction are inserted into the stack—and virtualization, in its simplest form, is another layer of abstraction—someone still has to manage the infrastructure. OK, so you’ve built a “private cloud” and you have highly virtualized infrastructure, pooled resources, self-service provisioning, etc. Someone still has to manage it. Someone still has to ensure that there is sufficient capacity, and that someone needs to understand the core technologies that make the private cloud tick. If we’re all moving “up the stack,” who’s left behind to manage the infrastructure?

This is why I’m not yet convinced that the age of the infrastructure engineer is over. Even if you have virtualized your servers, virtualized your network, and virtualized your storage, management of this infrastructure is still necessary. People who understand this infrastructure—both virtual and physical—are still necessary. Engineers who know the relationships among the virtualization layers and the various technologies are still necessary. Yes, the infrastructure engineer will change, grow, and evolve, but I think that the death of the infrastructure engineer is greatly exaggerated. We can’t all move up the stack into application development; someone has to stay behind and make sure everything runs the way it’s expected to run.

What do you think? Speak up and share your thoughts in the comments.

Tags: , ,

  1. Brandon Riley’s avatar

    I agree that the end is not exactly imminent, but I get the point. Will there be that many private clouds in 10 years? I think if you do have a private cloud, you’re gonna need an infrastructure team.

    The analysts would have you believe that 90% of companies will be running in the public cloud in the coming years. I think that’s the real question. Will that actually come to fruition? If so, then the infrastructure engineer will not die off, but will be an endangered species.

  2. John Dias’s avatar

    Scott you are spot on – anyone who thinks the infrastructure will become some sort of self-aware gelatinous blob that just needs power and AC is fooling themselves. The role of the infrastructure engineer will converge, as you point out, into a generalist instead of a highly specialized resource.

    What “cloud” is doing is adding automation and a layer of abstraction for the ultimate consumers of the resources built, managed and maintained by infrastructure folks. As for the developer folks working “upstream” from us in the DC, “cloud” is simply getting them out of our hair (actually out of each other’s hair).

    I do agree that it’s important that infrastructure engineers get a better handle on what’s happening at the app layer – but frankly that was good advice in the era previous to “cloud”!

  3. Jenu’s avatar

    Great post Scott. I think the main concern is for Engineers working with small to mid-sized businesses that will migrate their infrastructure to the public cloud–who do not have the means or the need for a private cloud. For this particular niche, and overall, there will be less demand for infrastructure engineers. However, I don’t think the role will become extinct since cloud providers and other large enterprises will still need someone who understands and can manage the core infrastructure

  4. Cam Case’s avatar

    Well written article. I would have to agree with you. As you said, “Awareness, though, is fundamentally different than being well-versed in application development.”

    It is important for anyone that works in IT to be aware of all aspects of IT. Developers should have an idea of how network and storage works and network engineers should have an “awareness” of developing.

  5. Steven’s avatar

    At first glance Scott this post worried me as these thoughts have been on my mind for a while now.

    I love me job and actually do see it dissapearing into the cloud (pardon the pun)

    However this post does actually make a point and that point refreshes me.

    Thanks for cheering my day :)

  6. Damian Karlson’s avatar

    I can definitely understand the viewpoint of the speaker at VMworld. The only reason that infrastructure exists is to support the “business” (in all of its forms). For years infrastructure folks only focused on their small, but important, area of the IT world – and that, along with ineffective or impotent IT management structures – has lead to a highly politicized, bureaucratic, and slow IT organization.

    The cloud revolution at its core, in my opinion, is to bring the business back into focus. The message is to simplify and streamline IT service delivery, and to deliver it in a way that’s meaningful to the business. Does Business Unit X care about the number of CPU cycles or whether their storage is multi-tiered or thin-provisioned? Usually, no. What they do care about is having an infrastructure that can meet their demand (flexibility & scalability), that it’s available when they need it, and that if something goes wrong at the application levels or higher (such as an outage or BC/DR event) – it can be made available (in the form it was previously) as soon as possible.

    The speaker’s point — that we’ll all be app folks sooner or later — is taking the “cloud” idea to its furthest (and probably most hyperbolized) point. I agree with you; there will always be a reason to have someone who understands how the nuts and bolts of IT. Perhaps the speaker needs to think about things from another point of view — take an automobile, for example. Most folks really don’t care that much about the inner workings of their car. Automatic transmissions, anti-lock brakes, and combustion engines are remarkable pieces of technology but most folks only think about their vehicles from a use case point of view. “I need to transport my family of 5 from point A to point B”, or “I need to be able to carry construction materials for my job”, etc. No matter how abstracted we get from the technology that makes a vehicle move quickly and safely, we still need mechanics and still need specialists for the more complicated pieces of technology in our cars.

  7. Stuart Miniman’s avatar

    Scott,
    I generally agree with you – VMworld had great information, but from Martiz’s comments (“infrastructure is boring”) to the heavy dose of cloud, infrastructure took a back seat. We know that infrastructure still matters and that somebody needs to architect and maintain a solution no matter when it physically lives. There may be a reduction in the number of infrastructure engineers who need to go deep and applications are the reason that we have infrastructure. These changes are going to take years and I’m sure there is still going to be a large audience to read your books and articles for a long time.

  8. James Klapperich’s avatar

    Perhaps eventually some of what we do today will become so common place that there won’t be a specialized skill set for it but I don’t think that’s a near term issue.

    It seems in my experience that while abstraction like virtualization smooths out the rough edges of the enduser/developer experience it actually introduces additional layers of complexity underneath that absolutely need to be managed appropriately which takes a special set of skills to do well. There has to be someone who understands all the magic going on to support the “cloud” to help figure things out when something doesn’t work exactly as expected.

  9. Justin Hensley’s avatar

    I’m trying to avoid a reactionary response here, as I wasn’t at VMworld this year so I obviously didn’t hear the talk. If that was indeed the conclusion that the speaker reached, I’d really like to question how they got up there in the first place.

    I think your point that skillsets are becoming more converged, just like our infrastructure, is much more on point. But that’s nothing new to be honest.

    The term “cloud” has been bastardized beyond recognition. People might be getting a little too fuzzy on it, in thinking it is some magical space where things just work. In reality, infrastructure engineers are the ones creating that “cloud” for developers/users.

  10. slowe’s avatar

    Thanks to everyone for their thoughts, I really appreciate you taking the time to comment. Keep the discussion going!

    Brandon, not only do we have to question whether public cloud adoption really will see the adoption curve that’s predicted, but we have to try to determine what sort of infrastructure the “public cloud” is going to require. Will it be vSphere-based? (VMware hopes so.) Or will it be open source, Xen or KVM? What about storage? Networking? Security? No one knows at this point.

    John, I agree—infrastructure folks have needed some awareness of what’s going on at the application layer for quite some time now!

    Jenu, see my comment above to Brandon. A lot depends on the adoption of public cloud and the architectures behind that adoption.

    Cam, agreed! Awareness at all levels of all levels is important, and long overdue. But what constitutes awareness? And how exactly does one build that awareness?

    Steven, I’m glad I could cheer you up!

    Damian, the car analogy is only partially applicable. To make it even more applicable, we’d need to apply the evolution of automobile technology to the mechanics. How have the mechanics evolved? Are there more mechanics who rely on automation and computerized tools now? Is that the equivalent of being “app developers” in the cloud era?

    Stu, my concern isn’t really about an audience to read my books, but you already knew that, didn’t you? :-)

    James, I agree that the introduction of virtualization does introduce some new complexities that require new skill sets and new types of expertise even as other skill sets and other types of expertise become less important. The real question for each and every one of us is this: how can I ensure that my skill sets and my expertise are still focused in the right areas?

    Justin, I don’t feel the speaker was/is unqualified; I think that, like Damian pointed out, he/she was simply taking this analogy to its farthest possible point, perhaps to add emphasis (I don’t know). Perhaps the point was simply to encourage infrastructure engineers to add that awareness of application development to their portfolio of knowledge.

  11. Aaron Delp’s avatar

    Hey Scott! I hope the move has gone well for you and Crystal!

    To your point, I see two sides of this. Private Cloud still needs an infrastructure engineer to run the “stuff” and probably an automation, orchestration, chargeback, showback person because the infrastructure engineer will have enough going on. So, in private cloud, there is actually an increase in people or an opportunity for an expanded skill set.

    In Private cloud you will see the opposite. The infrastructure will “go away” and the business now runs more on OpEx than CapEx.

    Which one will win? Who knows… As always it depends and there is no right universal answer, only an answer for each customer.

  12. Brian Gracely’s avatar

    Scott,

    Since you linked to my post, I thought I’d offer a clarification. All too often we take things to logical extremes, this case being no different. I think most would agree on a few things:

    - The traditional technology silos are blurring, especially in the infrastructure.
    - Just about everything you can do in hardware can also be done in software, in many cases at near feature-parity and close to similar scale.
    - We’re seeing more layers of abstraction via APIs (mostly around tools to manage)

    So my point was not that the infrastructure goes away or that anyone working in that space should drop those skills and become a programmer. But I do believe that people that do architecture-level jobs ought to have a level of knowledge about what is possible via APIs and what these new applications will do to the infrastructure. I never implied that anyone must become a developer.

  13. slowe’s avatar

    Aaron, I’m guessing you meant private cloud vs. public cloud (instead of using private cloud twice), in which case I agree. Should the use of cloud computing architectures and methodologies trend toward public cloud, then the availability of roles that focus on the infrastructure under the cloud will be constrained. Either way, the role of an infrastructure engineer is still necessary—it’s just a matter of who will employ the infrastructure engineers.

    Brian, thanks for your comment. You were not the speaker to whom I was referring, and I only used your post as an example of someone moving “up the stack” into application development. There’s certainly nothing wrong with that, and as I mentioned in the post—and everyone seems to agree here—there absolutely needs to be greater awareness of the application development process and the applications themselves. Personally, I applaud you for your efforts and wish you the best of luck! My thoughts were more directed at individuals who *are* implying that everyone must become a developer, and that’s not how I interpreted your post at all. Cool?

  14. Martin Glassborow’s avatar

    The best infrastructure engineers I’ve worked with are also pretty good developers. They automate and script using the tools and APIs available; they tend to be flexible about what they develop in.

    Even in the public cloud, you are still going to need infrastructure engineers; you might not call them infrastructure engineers but they will be working in very similar ways. They will be scripting automated provisioning and orchestration tools, they will be identifying bottlenecks and understanding the right way to build your virtual data-centre. They will be migrating between clouds, from private to public, private to private and public to public.

    And whilst we still have legacy applications; even running in the Cloud, they will be doing the job of making these resilient and robust. We are going to have legacy applications for some time…the job isn’t going away any time soon. My biggest concern is…where are the infrastructure engineers of the future going to come from?

  15. Jason’s avatar

    Working for a smaller environment and having done most of everything from data center design to P2V servers I think it is safe to say the days of infrastructure engineers are not numbered – quite the opposite. Having a well rounded skill set and familiarity with all aspects of what makes your SMB or campus run is a big tick in the job security column.

    Much like Brian states, the silos are transparent and merging… being a single person in a meeting and able to consider not only server resources and application response times, but san capacity, hvac, and power requirements, rack space, and lead time on hardware for the project – will make you a valuable position. Nothing VMware will sell you can replace that.

  16. Dave Lee’s avatar

    I’ve always worked for companies with numbers of IT people in single digits so I’ve always had to be a “jack of all trades, master of none” kind of person. Managing a virutal environment has just made me become a little more knowledgable in a few areas (mainly storage and networking). That said, I’ve always fiddled with scripting and a bit of web development so I’ve a good idea about how these things hang together. I think any successful IT person my this size of business will probably be the same so I don’t think this is as big a relavation to those in smaller businesses but probably more of a change for those in bigger companies where there is a lot more separation of duties..

  17. slowe’s avatar

    Martin, if we lump scripting into “application development,” then I would agree that “application development” is a key skill for an infrastructure engineer. I tend to view “application development” as something more than scripting, but that’s just my view. And I do agree with your question…where will the future infrastructure engineers come from? That’s a topic that might be worth another blog post…

    Jason, we all agree that skills are converging, and we all agree that awareness of the application development process is important. But what constitutes “awareness”? Is it learning how to develop? Or is it something else?

    Dave, I think that small companies might be affected here, too, especially if adoption of public cloud services takes off in that market, as many expect it to.

  18. Massimiliano Mortillaro’s avatar

    Hello. I was attending vSphere 5 and Cloud Infrastructure launch this week in Prague. We had a lot of presentations about how enterprises must simplify, how they will cut costs with the beginning of the Cloud Era. Everything revolves about moving to the cloud, simplifying the infrastructure (or getting rid of it for small businesses), cutting costs, reducing in house support.

    And yet I am not afraid of the future. Why? No matter how much you move from your own infrastructure to the cloud, nothing is running magically in wonderland. By moving to the cloud we are just delegating to a third party vendor and saying : “here, take my VMs, take my applications and make them run. I don’t want to hear about hardware, servers, redundancy, disk arrays or DR sites. Just make it run and make it highly available, and I’ll pay you for this”.

    That’s a nice argument for technology senior executives, who have a good argument to further reduce OPEX as well as FTEs. But at the end of the day, cloud or not, the business needs do not change. They need this app, they need it to perform well, without outages, and to be highly available. In the end of the day, this app has to be running somewhere, whether in-house, in a datacenter or in the cloud. And in the end of the day, there has to be some infrastructure, somewhere, to allow all of this cloud magic to happen, and most of all, some knowledgeable professionals who do support it. Here’s why I’m not afraid by the switch to siloed IT infrastructure to the cloud.

    As a system administrator, I have to deal with complex situations where the existing software isn’t exactly matching my needs, where the extra feature I need isn’t present. Sometimes such a solution will exist as a commercial product that fills the gap, but that will be either uneconomic to pursue, or the software won’t be allowed for use in my environment (welcome to large corp world). In this case, I have to rely on programming skills, no matter how weak they are, to come up with a solution that will allow me to fix the issue and to generate cost avoidances at the same time. In the development process, you’ll go through various iterations, through trial-and-error, through impossible headaches but in the end you’ll end up with a solid solution that works and exactly fits your need.

    Scott, if that’s the way you meant it, then I totally agree with your point of view.

    Disclaimer: I’m not a big league player and I’m not english native, therefore apologies in advance if something may be shocking!

  19. Jon Langemak’s avatar

    I too was at VMworld and as a network engineer; I was sort of taken back by some of the presentations. I heard one presenter say ‘We need to get rid of legacy technologies like spanning-tree’ (yes I’ve heard of L2 multipathing). While I understand that the network world and the virtualization world continue to merge paths I don’t see eye to eye with this kind of thinking. At some point, the virtual environment is still going to need to plug into my distribution or core switch so it can talk to other systems at other sites. It’s not like we can just deploy VMWare and plug an internet connection and a bunch of desktops into it. In reality, the virtual piece is only a small fraction of what most network engineers work on. There are still, IMHO, many pieces of a large corporations network that need to be managed by a dedicated team of network engineers. If anything, I think that this should be a call for teams to start working together in a more effective manner. At the end of the day, I shouldn’t be paged out for a virtual machine issue and Im sure the VM administrator doesn’t want to be paged for an OC192 going down. While I agree that it’s a good idea for there to be overlap (Im studying for my VCP) I don’t think, at least in large corporations, that these walls will ever come down. In regards to infrastructure folks being involved in app development, I cant see that ever happening.

    Just my two cents… Keep in mind, I work for a large corporation. In smaller shops, this might make more sense.

  20. Brian Masters’s avatar

    Some good insights on the prevailing theme at VMWorld, Scott. It being my first experience, there were a few times I didn’t feel completely welcome as an infrastructure/systems guy. From building the schedule to doing the labs, I got the feeling I was being pushed towards cloud as a resource and away from the nuts-and-bolts of building, maintaining, and optimizing one’s virtual infrastructure. I can understand the push from the business side, but it’s reassuring to hear your rebuttal.

    While I initially discounted Brian’s assertion, it does seem like a question we need to be thinking about. The transition from (storage, network, VI) specialist to generalist is built on the simplification and abstraction of those pieces away from their most basic architecture. In so doing, there is a commensurate loss of efficiency and optimization in those systems that we deem acceptable. This would lead to the question, at what point can any given virtual infrastructure be over-built and over-provisioned out of the box such that it no longer becomes important to wring all you can from the underlying resource?

    As Moore’s Law marches on, this point approaches for larger and less-specific systems, and the value of any given infrastructure engineer will depend increasingly on his ability to manage many hundreds or thousands of systems at once (i.e. scripting and custom development).

  21. Mike Barratt’s avatar

    Thoroughly enjoying this post and the excellent comments.

    I agree with Jason, being able to sit in a meeting with a client (or the ‘business’) and being able to offer valuable input in several areas is priceless. However, this clearly can’t be gained overnight and takes years of experience.

    Never forget the end goal, which is making IT as flexible as possible to fulfil the current and future needs of the business, often at the lowest cost possible. Even taking cloud computing out of the loop, as an employee for a global Managed Services provider, we have picked up an awful lot of work from traditional companies who had multiple infrastructure teams (servers/storage etc) and you can bet your bottom dollar it’s being done with far less people who manage multiple clients and need to be well versed in different areas. The scope is always increasing too, we need to know a lot about SQL, SharePoint etc which brings me back to my earlier comment – to be effective, the level of knowledge and experience a modern day Infrastructure Engineer needs takes a long, long time with multiple successes (and failures) along the way. If you’re only able to configure a server, or provision a LUN, you may well find yourself having a hard time competing against others for a job with a cloud provider in the future. I wouldn’t like to be starting out fresh in infrastructure now that’s for sure, or perhaps IT in general? Not sure……..

    Like you said Scott, I’m also not so sure on the Application Development point. Sure, any knowledge an Infrastructure Engineer has in development can only be beneficial but personally, for any junior Infrastructure bods out there I’d be concentrating on getting well versed in as many Infrastructure technologies as possible, then bolt on some common applications on top. Someone still has to run all this stuff.

  22. Martin Glassborow’s avatar

    If you look at a lot of development which is carried out in an agile manner; there is more in common at times with scripting than some traditional development methodologies.

    And when I talk about scripting; these are guys who are writing Python, Ruby as well as the more traditional scripting languages. Code re-use is pretty common as well; write one script and modify it to suit. Perhaps they are Ops-Devs…as opposed to Dev-Ops.

  23. ChaddD’s avatar

    Great post again, Scott!

    So two points I would make to argue against the extinction of the Infrastructure Engineer. As you say someone still has to manage the infrastructure. Whether this infrastructure is in house at the SMB or sitting out in a datacenter there is still the certainty that someone needs to setup this hardware, monitor, and fix it when it fails. If a business were to buy a bunch of preconfigured hosts, have the vCenter VA sitting on it, and roll in a VNX5700 someone still did the configuration and someone still has to setup this environment at the site.

    During the general session Paul noted that he was glad that he didn’t get stuck in the mainframe era and that his working with software was a blessing in disguise. I think its fair to compare the mainframe managers to the networking, storage, etc specialized teams that are fading now, but not necessarily the Virtual Infrastructure System Administrators we are evolving (have) into now. Already I see developers in my own environment keep available resources in AWS just in case we can’t respond to requests quickly enough. As Sys Admins I do think we need to be conscious of what the users of our environments want and need so we can provide them, or empower them to get, the resources they need in a timely manner in house. Infrastructure Engineers going extinct, no, specialized teams, probably.

  24. Lisa Caywood’s avatar

    Scott, this is a pet topic of mine, and I agree for the most part with your comments and those of others, most especially Messrs. Gracely and Delp (though it’s hard to imagine how infrastructure magically “goes away” in a public cloud scenario–it’s just in a different place.)

    The person who probably does need some dev skills, though, is the automation & orchestration person Aaron talks about. Not northbound to business apps, except to the degree that s/he should be able to comprehend app architectures and how they impact infrastructure and be able to articulate this information to app dev teams. But rather southbound, in order to broker smoother interaction between infrastructure domains and their respective management stacks. This is a highly specific (and relatively new) type of role, however, and not necessarily of interest to most infrastructure engineers. And that’s okay. Deep domain expertise will continue to have its place too (I like the “sliding partitions” image), although I do think the majority of infrastructure roles will become increasingly generalist in nature.

  25. Allan Robertson’s avatar

    I guess like every other role in IT over the years, the Infrastructure Engineer will evolve.

    Today they are saying the say the cloud will kill off the Infrastructure Engineer and NoSql will kill the rdbms.

    10 years ago, Oracle 9i, the “self-tuning” database was going to kill off the DBA. Many still seem to be making a nice living at it.

    No doubt technologies will change as their popularity rises and falls and the engineers, administrators and developers will adapt or move on. For example, in the 90s Netware was a pervasive technology in corporate IT. Times, technologies and vendors change. So what happened to all those Novell experts, did they die out like the dinosaurs or did they reskill in networking, MS and VMware and move on?

    Undoubtedly, the cloud will offer new opportunites that haven’t been guessed at yet. The extinction of the Infrastructure Engineer may herald in the rise of the cloud lawyer as customers sue for missed SLAs and lost data as environments have insufficient protection, backups and untested DR. Before everyone moves up the stack they had better make sure the foundations are there to support them.

  26. Dan Gordon’s avatar

    I’ve always thought there were systems people and apps people. the apps people love the “last mile” to the user. the systems people love to optimize a well-defined (or even not-so-well-defined) system.

    Scott properly says that systems people will have to understand more about apps to do better systems work, but that the two will remain somewhat distinct.

    IMHO

  27. RedSneakers18’s avatar

    Very good insight Scott, I have also realized this unavoidable series of events for quite some time while making (forced) transition to some *aaS. The many layers of abstraction have sent a shot across the bow for the infrastructure engineer. Case in point are we not working ourselves out of a job? How many Infrastructure Engineers will actually be needed in the future when the key selling points are to minimalize hardware and resources. How many certifications will be required to keep a job when the reduction in force narrow down both opportunities and the available positions. There will always be small and mid-size businesses that will require a staff of Network /System Admins to keep the business running during the migration to leased Cloud Infrastructure for the forseeable future. Just saying, the Enterprise corporations are currently putting in the infrastructure and will realize the promise ease of expansion from the Cloud, whether they own it or buy leased services. I am hopeful that new opportunities will be child out from the Parent Cloud adoption process that are currently unforseen now but will be realized going forward.

  28. Jake Blakes’s avatar

    Sure, VMWare will say the infrastructure is boring , what would you expect? They don’t sell network, storage, compute, racks, etc. that make up infrastructure. The sell software, and vmworld is their platform to sell their stuff. Ask the same question to Cisco, IBM, EMC, and I bet you will get a different answer.

  29. Keith’s avatar

    This is an interesting topic and article and after reading all the comments the quote that Justin made is so true, I see people using the term Cloud now for just about anything because it might sell better. It’s the future, I get that no problem, but the term Cloud computing is out of control.

    Who is managing all this Cloud infrastructure? Cloud BOTS I mean if it’s done correctly the Cloud by itself won’t patch or resolve problems… will it? I don’t see the infra engineer role coming to end, but rather evolving into perhaps the Cloud Engineer.

    Mmm… maybe there are just to many Severs in the Sky with Diamonds at the moment…..

  30. Darren’s avatar

    … hmmm well, I guess I won’t need to buy your new book then!

  31. Iñigo’s avatar

    Disagree about disappearing.

    I’ve never known “separated” infrastructure teams, maybe because I work in a company whose main line is not system administration.

    But totally disagree about disappearing. The cloud just change contract rules.

    Someone needs to know about security, drivers, OS, storage, backup, networking, etc to create a successful hosted infrastructure. Being with cloud contracts or traditional ones.

  32. Mimmus’s avatar

    What you say in the article is also my feeling: unfortunately, IT is not an intrinsic value and business is always ready to escape where its processes can be satisfied with lesser cost.
    If middle and small companies will escape towards public or external clouds, jobs like infrastructure engineer will decrease in number. Cloud providers will not be so many and concentrated in large economic districts. Finding a good job will be very difficult and only in large cities, where life quality is worse.

    But peraphs it is only a bad dream, “cloud” will be only a big bubble like “outsourcing” some time ago :)

  33. Jivetolkein’s avatar

    When I first moved into infrastructure some time ago, I was offered a valuable insight into problem management, that still holds true..

    A customer has an issue. They call the service desk.

    The service desk can’t help. They escalate to desktop services (B level PC support).

    B level can’t help. They escalate to the application owners/responsibles.

    They too are stcuk. They escalate to the app developers.

    They scratch their heads. They hmm and haar a bit. They send it over to the infrastructure guys.

    The infrastructure team look puzzled. They turn around to see what they can do with the problem.. and just see a big pile of manuals..

    {n.b. – this was before the glory of Google ;-) }

    Point is, there is always a last point of call, and in my experience it’s usually the ‘Infrastructure team’ – they get stuck with owning it one way or another :-) – these are the employees you don’t want to lose.

    Multi discipline knowledge is already a given for more or less any role these days, but you still need the specialists – I always like to sanity check my vnetwork changes with the guys who spend all day in IOS, it’s just the prudent thing to do.

  34. Julian Wood’s avatar

    There is certainly benefit in everyone being more multi-skilled and infrastructure people learning more about the application stack. That doesn’t mean they need to be developers but rather have insight into the platform layer which is going to be the foundation for IT and enabler for true cloud computing rather than the infrastructure of VMs we have at the moment.

    However, until we are there which is going to take many many years I actually think we should be looking at the inverse. Application developers should be moving down the stack and care more about the infrastructure components to build highly available and dynamic applications.

    Why should functions like high availability & DR be only a function of the infrastructure layer invisible to applications which requires maintaining replication / clustering / mirroring / FT etc. technology to give you high availability.

    Applications should be written to be self recoverable, highly availably without requiring clever infrastructure. Move the availability up the stack!

    This reduces costly infrastructure as your hosts/servers/VMs become worker nodes and could even run on local storage anywhere. Google is arguably the worldls most scalable cloud computing environment and it doesn’t need VMs!

    Once developers are required to write applications with availability in mind and design their applications to fail rather than relying on the infrastructure to compensate for non dynamic applications, then we’re really moving towards cloud computing.

  35. Adam Martin’s avatar

    I agree that we as IT engineers need to continue to expand our skill set and not become stagnant. I think this is the key to having a successful career. However, I don’t think that it means that everyone needs to become a developer though either. I am not sure that is what they guy was trying to say, but I could see how some would take it that way.

    I do think it is important to have a “good” understanding of the different technologies/products that are helping run the business and that interface with your area of expertise. I do think the days of the the “networking team”, “storage team”, etc are slowly going away. Everything is blending and I feel that it will only continue to do so even more in the future.

    Regarding the comment about the majority of SMBs moving to the public cloud, I am not sure about that. I work heavily in that sector and there is still a lot of resistance to that and for good reason based on the requirements of the business.

    As always, great article Scott. Just my two cents.

  36. Ben’s avatar

    There’s no doubt that managing infrastructures is becoming less tedious and requires more expertise, thus reducing the number of infrastructure engineers needed. But saying that they will be eliminated is like proclaiming that natural language programing languages will eliminate developers.

  37. Brian Gracely’s avatar

    An interesting blog from the Facebook “Infrastructure Engineering”
    team – http://www.facebook.com/notes/facebook-engineering/making-facebook-self-healing/10150275248698920

    I know lots of people like to say that this is an outlier, but it’s
    interesting to see if there are things to be learned by the
    people doing things at massive scale.

  38. John Gill’s avatar

    Optimistic idea, but I think you would be well served to be the guy that was at this confluence of skills, but even people within a given area are still commonly un-expert. I don’t see a lot of people becoming more rounded or better, just a position where if you do know these things, you will be well off and secure in your career path.

  39. ChrisW’s avatar

    Really interesting article and some great replies.
    I would like to add that alot of the points only seem to apply to larger companies. All of the SMBs I’ve worked in (which isn’t many, granted) have had small teams that cover every part of the infrastructure already. You do tend to get certain people who gravitate towards certain roles but they are well versed in all of the others as well.
    As the SMB-Large Enterprise barrier is breached and the infrastructure expands the roles start to separate. The large enterprises I’ve worked in still do have separate teams for each technology. In those environments it would have been almost impossible for the roles to merge to fully because of the scale of the operation. It that scenario it was key that the teams worked well together (thankfully they did for the most part).
    In both types of organisations there will be sections of the infrastructure that can be hived off and sent ‘to the cloud’, there will be others that can’t. And then there will be businesses which simply can’t send any data ‘to the cloud’ because of their nature (dealing with sensitive data for example).
    So is the infrastructure engineer role dead? In my opinion, not for a while yet and possible not ever. There may be a reduction in certain sectors but others will be largely unaffected.

  40. Michael Wolcott’s avatar

    I think this awareness and discussion is highly valuable. However, worrying about any of it is probably taking it too far. *aaS is what IT has been about since day one. It’s just the definition and delivery of that service that’s evolved — but we’ve all evolved with it. And we will continue to evolve with it into the future.

    Call it reskilling, call it converging specialties, call it anything you want. At the end of the day, it’s all about always learning something new and staying current in our industry. I have no doubt there will continue to be enough new things to learn that we’ll find an area that we enjoy enough to excel at it and stay in the business of *aaS, regardless of whatever term or buzzword the industry uses to describe it. The only obsolete engineer is the one who is unwilling to adapt and grow. But that’s the way it’s always been in IT. The percentage of companies that move to a public cloud is irrelevant to our constant need to adapt and grow except that it’s another example of why it’s true.

    As for where the next generation of Infrastructure Engineers will come from, I’d say mostly from the same places as before. As infrastructure evolves, internal helpdesks evolve with it, as do college curriculums, technology training platforms, and even consumer products and the way we use them. I doubt many of us worked into the skill sets we have today without some type of mentoring or training, and we certainly weren’t experts when we started. I can’t imagine the future will be different in that regard — there will always be a mix of novices and experts across organizations, the one hopefully learning from the other.

    Good post, and good discussion. I’ve enjoyed it — sorry I didn’t get a chance to read it earlier!

  41. Matt’s avatar

    You can definitely see how IT professionals are becoming more adapt at working inside differing concentrations within the field. Having said that, I think it can only be done to a point. It is hard to become an expert at anything if you do not concentrate on anything specific. For high level enterprise environments, I still think you need people that are specialized in a given area – storage, network, database, etc.

    But to add into all of the application development, I think that is a bit of a stretch. We’ll see, though.

  42. Alex Witherspoon’s avatar

    I would say the role is anything but gone: http://aws.amazon.com/careers/

    Though, you might find yourself working for the mega centralized data center instead of directly for the consumer if companies choose to outsource.

  43. Simon Hamilton-Wilkes’s avatar

    Lots of salient comments, I think the gist is nothing will change in the short to medium term, but maybe in ten years Infrastructure engineers will be a similar number to now, rather than have grown.
    I think one of the key things is the number of failure modes of a current cloud is much greater than that in conventional enterprise computing; there’s going to be a whole new set of requirements for VCDX type folks who also understand active/active, and scale out applications and distributed DNS etc. There’s also the access layer issue – where BGP multi-homing or other redundancy becomes more critical still when everything is hosted off-site.
    In the utility computing, cloud nirvana where everything is virtually hosted the data center becomes very dynamic/agile, which is going to be great for business, but also require plenty of us infrastructure folk to architect, deploy, maintain, and fix.

  44. Doug Pardue’s avatar

    I didn’t read through all the posts, so I may be echoing someone else’s thought…

    But one possibility is that each of our brethren (and “sistren”?) who have spent years honing their respective crafts, may just have to learn to work and play together better. I would liken it to some of the development models of recent years, where programmers work in pairs or small teams, both eyeballing the same unit of code, collaboration its development.

    Likewise we will become more development-minded as we script everything we can to keep our clouds automated, scalable and predictable (stable).

  45. Donny Parrott’s avatar

    Interesting topic. If you stop and consider the roadmap, a few things lend credence to the limitation of opportunities for infrastructure personnel.

    I believe another outfall from [cough] ‘cloud’ [cough] service delivery is migration of intelligence to the edge. As systems become stateless and rapidly dynamic, the infrastructure design must simplify. I believe this is what Maritz is referencing.

    I see a future when vCenter (or another orchestrater) will configure all devices. Each silo will contain only tier one personnel as the orchestration personnel will be responsible for all configuration and monitoring. An envrionment where a new physical swith is racked, given a management IP, and registered within the ‘orchestrator’. Networks, security profiles, etc. will all be orchestrated on the fly. There are already designs being researched where everything behind the firewall is virtual.

    So as the architecture becomes more virtual and capable of orchestration, the tier 2/3 admin and engineer may be under the greatest risk.

    I believe the future growth is in service delivery engineering. This individual will take the business requirements and define the resources necessary and coordinate their delivery to the SDM responsible for application deployment.

    The other large growth area will be in resource management. The amount of orver-sight and vendor management required for “on-demand” IT will drive acquisition and resource management workloads up.

  46. Mike Jordan’s avatar

    I had the very same conversation with someone at VMworld. My perspective is that the Infrastructure Engineer will not disappear but it will have to morph. As all technologies become more advanced, the idea of someone being an effective “swiss-army knife” becomes less and less plausible. It will not be possible say, be great at VMware without advanced knowledge of networking and storage as well. For instance, would it be possible for a single person to stand up a UCS chassis, Nexus switches, and a VNX storage platform all on their own? And be able to implement the best practices for security, performance, and scalability for each? All of that before you even talk about the VMware part of the stack and knowing how to design it all to work together. Other than the select few bad-asses in the field that do this day in and day out, it just does not seem possible to expect so many other to be able to do the same.

    In the end, the Infrastructure Engineer will still be necessary. His role will become more of a technology manager that brings together the talent from the different disciplines. Knowing who and when to call on as well as being able to translate ideas between groups and calling out the BS will be his new role. Plus everything he was doing before :)

  47. Larry’s avatar

    The speaker is pushing VMware and their Cloud offerings…..to make money. So his spin is of course going say such things.

    In “REAL WORLD” it is just a line of BS. Personally I am getting tired of the over used “CLOUD” term.

    It is not a new concept at all. Corporations have been using the cloud for 10 years. I worked at a large company 10 years ago when the outsourced all of our benifits to the “cloud” complete the the tiny call center that we had internally. So what, we shut off 2-4 servers maybe.

    Consumers have been using the cloud as well for many years…..hotmail anyone?

    The cloud has its place but often, with bandwidth and other costs the cloud is not the best solution. VMware, Microsoft and others are jamming it up our arses because they need new revenue streams. Dumb CIO’s embrace it to make them look good, thinking they are with the new stuff.

    Lastly to the point of this speaker, if virtualization brings anything into the IT world that is complexity. The “Infrastructure Engineer” job is harder, more complex and finding peopel that “get it all” (VM, storage, networking) is very hard to do. We have had open posistions for months because we simply cant find people with the right skills.

  48. Alfred’s avatar

    The advancement of technology has always made some jobs redundant. The infrastructure engineer will still be around but may be in lesser numbers.

    Just like in IT outsourcing. Some companies outsourced everything, but in the end found it to be more cost effective to do the new stuff in house.

  49. Gary Huber’s avatar

    Want to know what’s going to happen to Infrastructure Engineers? Between the cloud and overseas outsourcing, they’ll be nearly extinct in 5 years.

    That’s my .02 cents. Just look at the trends — it’s scary.

  50. slowe’s avatar

    Gary, who’s going to build/monitor/maintain/optimize “the cloud”? Skynet?

1 · 2 ·