First, my heart goes out to out to all those who have found themselves out of a job during these dark financial times. I hope you won't find this post antagonistic in any way, it is obviously written from the perspective of someone who is still gainfully employed in a large enterprise (but what the future holds is anyone's guess). If you are in a tight situation right now, I can only hope that this post will give you some ammunition for your next interview, or help give some focus to your time between jobs.
OK, I'll lighten up a little now :-). So the title of this post may seem a little odd. All we seem to be hearing about these days is lay offs and cutbacks. I know of several institutions who have mandated all contractors to not be renewed, and others who are chopping full time staff alongside contractors. Either way, doesn't sound too great at all, and in fact I'm beginning to think I may change that title.
But then I remember what's great about restriction - it's a lightning rod for innovation. Look at all the success stories on the web - the vast majority of them were born from tight circumstances, from the poor student to the unemployed developer. And if you're on top of your game in the enterprise, it's no different - you can write your own internal success story.
Some of the things I hear around my office are no doubt typical of any architecture & engineering group in a large investment bank. The focus from on high is on keeping the lights on, and cutting costs at any available opportunity. But what's maybe different is the approach to cost cutting, which is absolutely grounded in fundamental mathematical principles. What do I mean by that? If you need to spend $1 to save $2, then you're still saving $1 so it's worth doing.
But why would you need financial dire straits for such behaviour? Surely you would follow the same principles in the good times as well as bad? Damn straight we do - the difference is in these times, _everything_ can be challenged. And when you can challenge everything, you can innovate like an engineer possessed.
Case in point, clouds. And let's be honest, there's only one viable player in the market currently, Amazon. And boy are the eyes of the business on them. But what's been lacking until now is the ability for an internal cloud to compete with them on purely a cost basis. Amazon's EC2 pricing includes a bucket of compute resources, SLA's for those compute resources, and a choice of preconfigured AMI's (I'm conveniently ignoring the ability to roll your own AMI and upload it, for the time being).
Here's what Amazon's pricing doesn't include. Guest support, backup, monitoring, antivirus, patching, and auditing. Where is the greatest cost associated with enterprise compute resources? Those very same things.
Now, yes I am conveniently ignoring the ability to roll your own AMI and upload it to EC2, but even if you did include all the aforementioned agents in your image, how practical would that actually be? I have yet to see an enterprise with a monitoring system that was intelligent enough to know the difference between "down because someone shut me down because they don't need me right now" and "down because of a hardware / software error". So if, in order to get Amazon's cheap pricing, we're willing to forgo monitoring within the guest, then surely I can do the same for my internal cloud machines? And the same goes for backup. Machines on a backup schedule don't only attract backup software licensing costs, there is monitoring overheads, and storage costs associated with the backed up data. And again, backup systems are generally inflexible. A backup missed is a backup missed, and generally the operational decree will be that it needs to be run at the first opportunity. But in order to get Amazon's cheap pricing (there's additional charges for network IO), we're willing to forgo backups. Hey, there's another cost I can strike off the list for my internal cloud offering! How about authentication? Domain membership has all sorts of implications for creation / deletion / archiving or machines, snapshots that get rolled back beyond 30 days etc. Windows based EC2 machines with "authentication services" attract something like a 50% premium. Perhaps my internal cloud machines should too. After all, does every development box _really_ need to be on the domain? How about grid nodes?
I won't bore with going through the rest of the list, but you get the picture. In order to get this rock bottom pricing, there is a _lot_ of functionality that is generally taken for granted in the enterprise that needs to be stripped out.
One of the biggest challenges in the enterprise is reworking the charging model to strip out all these things that now look like "extras" in comparison to Amazon's EC2. And this is where the "great times for enterprise computing" comes into the picture - it's about time these things were fucking well treated as extras, and that an _accurate_ pricing model was available for the business that broke down all these costs, and made them optional. That's the only way we'll get an apples to apples comparison to the likes of EC2.
And you know what? It's actually happening where I work. All the restrictions that would've hamstrung us from ever offering something on par with EC2 are being lifted. Finally, cheap utility _compute_ (I can't stress that enough - COMPUTE) is something we're actually going to be able to offer for the first time ever. And it's all thanks to the financial crisis, because the laser like focus on cost would never have happened otherwise.
This of course also has implications for VMware. When I say _everything_ can be challenged, I mean _everything_. Would it be cheaper for us to pay for VMware, or for someone like me to be given a few internal developers and infrastructure resources and take a shot at building something like EC2, right down to the Xen part of things?
Right now in many companies, it's survival of the fittest. If all we're going to be left with internally is a skeletal staff of absolute guns in their respective fields, along with a mandate to drive down costs through innovation and to hell with how things used to be done, then you can bet your bottom dollar something like that is entirely possible.
So if you do find yourself out of work, maybe it's time to further develop those automation skills. Get familiar with web services, pick any language you like - C#, Java, Python etc. PowerShell is a great option too, check out the PowerShell 2.0 CTP. Sign up for an Amazon Web Services account and figure out how to do stuff (you only pay for what you use, it can be cheap). Think about what they offer, find the strengths and the weaknesses, and then think about you might implement something similar in an enterprise, and what would be required to do it better. Think about how you might burst into something like EC2 from an internal cloud. What layers need to be loosely coupled in order to do such a thing in the most efficient way? What implications do external clouds have for internal cloud architecture and operations? Now take all that, and expand your scope to Google's cloud offering, what's coming with Azure, what other players there are in the field.
Yes sir, exciting times ahead in the next few years, even moreso than they were before the financial meltdown occurred.