I really need to bite the bullet on this one and move to my new wordpress based site. In the next few hours I will revert the current site to it's default vinternals.blogspot.com address, so the RSS feeds should still work. Then later this evening I'll point the vinternals.com domain at the new site. Finally, in the next day or 2 I'll burn the feeds from this site. If you haven't seen anything from me by next Sunday, either it's all gone horribly wrong or you need to update your RSS bookmark.
Hmmm, it seems putting the word burn all over the place has resulted in ads for "sexy singles" to temporarily appear on the site. Will have to use that word more often :D
Sunday 15 February 2009
Saturday 14 February 2009
Getting from Nice to Cannes / Cannes to Nice
VMworld Europe 2009 is only 1 week away, woohoo! I'm certainly getting pretty excited about it, so I thought I'd post up some info for anyone who is getting into Nice outside of the VMware provided bus times and travelling on their own dime or on restricted company dime (times are tough, I suspect most of us are lucky to be going at all).
The buses services around the Cote d'Azur are great - I've taken them a few times during various jaunts to the area and been amazed at what you get for a basic 1€ ticket. From Nice Airport, you have 2 options for buses - the 1€ Cannes RN7 (bus number 200, around 90mins journey time along the coast) or the ~15€ "Aeroport Express" (bus number 210, around 50mins journey time via the highway).
Here is a link to the timetable, route and departure location info for all buses departing from Nice Aeroport (might be worth saving the Bus 200 and Bus 210 pdf's to your phone / laptop / ipod). Below is a screeny from the Nice Aeroport website with some basic info on the Bus services to Cannes.
UPDATE Hit the comments for direct links to the Bus 200 timetable and route info PDF. Thanks Anonymous, and sorry to hear you won't be there.
The buses services around the Cote d'Azur are great - I've taken them a few times during various jaunts to the area and been amazed at what you get for a basic 1€ ticket. From Nice Airport, you have 2 options for buses - the 1€ Cannes RN7 (bus number 200, around 90mins journey time along the coast) or the ~15€ "Aeroport Express" (bus number 210, around 50mins journey time via the highway).
Here is a link to the timetable, route and departure location info for all buses departing from Nice Aeroport (might be worth saving the Bus 200 and Bus 210 pdf's to your phone / laptop / ipod). Below is a screeny from the Nice Aeroport website with some basic info on the Bus services to Cannes.
UPDATE Hit the comments for direct links to the Bus 200 timetable and route info PDF. Thanks Anonymous, and sorry to hear you won't be there.
VMworld Europe 2009 - "vinternals" Will _Not_ Be There...
... but Shyam and Stu from big-european-investment-bank-that-has-nothing-to-do-with-vinternals will be. Why am I saying this? Because it would be irresponsible of us not to have some dicussions with various vendors regarding their roadmaps and other potentially sensitive information during the course of the event. We completely understand that such discussions are not possible with media-blogger-types.
So to anyone out there who we do talk to, rest assured that the corporate non disclosure "cone of silence" will be in full effect, any information disclosed under NDA or the like will not make it's way into the public domain via us. We are _not_ attending VMworld as "vinternals", we are attending as responsible corporate citizens. At least while we're sober ;-)
So to anyone out there who we do talk to, rest assured that the corporate non disclosure "cone of silence" will be in full effect, any information disclosed under NDA or the like will not make it's way into the public domain via us. We are _not_ attending VMworld as "vinternals", we are attending as responsible corporate citizens. At least while we're sober ;-)
Wednesday 11 February 2009
ESXi Health Status Inequality - HP vs Dell
Ok, so I'm pretty sure most of you know I've been haxing around with ESXi for quite some time, but I only recently noticed the difference in what you see in vCenter Health Status depending on which hardware vendor you use. I'm sure you also all know that I'm a massive fan of HP, which is why I'm a so unimpressed with their integration with vCenter. For example, lets have a look at recent generation HP BL685c blade running the HP OEM version of ESXi 3.5 (installable), downloaded from the VMware website.
Notice anything strange? Or rather, notice the distinct _lack_ of anything out of the ordinary. That's right, when comparing this blade to a non-OEM ESXi installable, it looks exactly the same in vCenter! At first I thought there must have been a corruption in the CIM providers or something, but after investigating with my VMware SE, we could find nothing out of order.
Now compare the image above to what you see in vCenter on an old Dell rackmount (note the bios date - nearly 2 years old!).
Wow.
Now I can only hazard a guess that HP want you to use Insight Manager to perform health monitoring exclusively, but why force me to deploy more monitoring infrastructure than I need to? They could at least create a VI client plugin for Insight Manager so I can use a single pane of glass! But even with that, it doesn't seem like you get as much information as what you do with Dell - looking at Appendix B of this HP whitepaper, storage subsystem information is strangely absent.
If anyone else out there has seen this, seen different, or can compare with other hardware platforms like Sun or IBM, please share your experiences in the comments!
Notice anything strange? Or rather, notice the distinct _lack_ of anything out of the ordinary. That's right, when comparing this blade to a non-OEM ESXi installable, it looks exactly the same in vCenter! At first I thought there must have been a corruption in the CIM providers or something, but after investigating with my VMware SE, we could find nothing out of order.
Now compare the image above to what you see in vCenter on an old Dell rackmount (note the bios date - nearly 2 years old!).
Wow.
Now I can only hazard a guess that HP want you to use Insight Manager to perform health monitoring exclusively, but why force me to deploy more monitoring infrastructure than I need to? They could at least create a VI client plugin for Insight Manager so I can use a single pane of glass! But even with that, it doesn't seem like you get as much information as what you do with Dell - looking at Appendix B of this HP whitepaper, storage subsystem information is strangely absent.
If anyone else out there has seen this, seen different, or can compare with other hardware platforms like Sun or IBM, please share your experiences in the comments!
Monday 9 February 2009
PowerCLI Mastery, Volume 3
Phew, Volume 1 and Volume 2 down, lets wrap this monster up. See, I told you I wasn't going to leave you hanging for weeks to nail through all this.
So, picking up right from where we left off. We need to perform a configuration operation on an ESX host. Of the available properties we're looking at now, which is most likely to tell us what we can configure? That's right, the configManager property! If you guessed "config", you were close - the config property tells us _what_ the current configuration values are. The configManager object tells _how_ we can do the configuring (is that even a word?). Lets click on that.
Now we're cookin! AS you can see at the top of the page, the configManager object is a Data Object of Type 'HostConfigManager', the properties of which contain a whole bunch of Managed Objects. The task at hand is to modify a portgroup configuration, so let’s check out the 'networkSystem' property.
Here we see all the available properties and methods available for the HostNetworkSystem Managed Object. And low and behold, down the bottom we see the method by which we can modify a portgroup, UpdatePortGroup. We can see that 'networkSystem' is actually a Managed object, but we have to traverse the configManager _Data_ object to get there (because we need to provide the correct context for the managed object reference). Recalling that Get-View can only return _Managed_ Object references, in order to get the HostNetworkSystem Managed Object reference for our particular host, we have to again use Get-View but this time passing it in the networkSystem property of the configManager Data object. Which is a lot easier in PowerShell than it is in English.
OK, finally we see that in order to use the UpdatePortGroup method of a HostNetworkSystem Managed Object, we need to pass in a portgroup name as a string, and a portgroup as a... HostPortGroupSpec object??? WTF!!! Stu, you lead us all this way to a dead fucking end!!! There won't be anything left for Boche to mash at VMworld Europe 2009 by the time I'm done with you!!!
Breathe. Stay Calm. Focus. I have only given you 2 of the 3 tools needed for PowerCLI mastery. Now, we turn to the dark side... err I mean we turn to the SDK documentation, version 2.5 of which is online here.
The online raw SDK documentation is the equivalent of that goo they eat for breakfast in The Matrix - it contains all the things a PowerCLI Master's body needs. You can have a read of the text in the main window there if you want to get a much better explanation of the different object types that what I've given, or you can just jump in, find what you need, then get the fuck out. Yes, while the raw SDK documentation contains everything the body needs, reading it is also a bit like staring at the Sun. Do it for too long, and you'll go insane like that guy in Sunshine.
Open up the online SDK documentation site. Recalling our conversation about the types of object in the VI API, you can reasonably deduce that a 'HostPortGroupSpec' object is likely a Data object. So click on the "Data Object Types" link in the top left hand corner, wait for the list underneath to update then scroll down to find "HostPortGroupSpec", and click on it. This is the equivalent of the part of our PowerShell code where we do:
Now where did I pull that “VMware.Vim.” from? We are using a native .net assembly given to us by the VI Toolkit, Vmware.Vim.dll. You probably haven't realised this up until now, I certainly didn’t until about a year ago, but the VI Toolkit is not just PowerShell - it's actually also a native .net wrapper for the VI API, that you can use with C# instead of compiling your own proxy dll from the VI web service. But you don't need to know the nuts and bolts of that - this is something I'm going to ask you to take on faith. You will eat your greens, you will look both ways when crossing the road, and you will know that the VMware.Vim.dll contains a reference for _every_ object available in the underlying API and then some. And for those sceptics out there who don't believe me, execute the following in any old PowerShell window on a box that you have PowerCLI installed on (and make sure u have a large row buffer ;)
Back to the SDK documentation and the code. Now we see what the properties of a HostPortGroupSpec are. There are 4, and all are mandatory. But what’s this, one of those properties is _another_ object, a "HostNetworkPolicy". So what do we need to populate one of those? Click on it and we'll see!
Now you understand why I'm such a huge fan of PowerCLI. I've had the displeasure of coding against the VI API directly in C#, and Shyam did most of the heavy lifting in Java when we developed statelesx. It often seems convoluted and contradictory, you often find that what you thought would be a simple operation based on another seemingly similar task, is actually anything but. As you've seen in this example, there are layers upon layers upon layers, you can get lost pretty easy. But thanks to the foresight of the VI Toolkit team to expose the underlying VI API via Get-View and provide us with VMware.Vim.dll assembly, anything is possible with the Power of Shanklin. Ooops I mean the Power of Shell ;-). But the Power of Shanklin does compel you.
Hopefully now you can not only do whatever you want to with PowerCLI, but also look at any script that the forum wizards may post, and understand exactly what they’ve done and how you might tweak them to your needs.
Now go forth and conquer. You are Jedi.
So, picking up right from where we left off. We need to perform a configuration operation on an ESX host. Of the available properties we're looking at now, which is most likely to tell us what we can configure? That's right, the configManager property! If you guessed "config", you were close - the config property tells us _what_ the current configuration values are. The configManager object tells _how_ we can do the configuring (is that even a word?). Lets click on that.
Now we're cookin! AS you can see at the top of the page, the configManager object is a Data Object of Type 'HostConfigManager', the properties of which contain a whole bunch of Managed Objects. The task at hand is to modify a portgroup configuration, so let’s check out the 'networkSystem' property.
Here we see all the available properties and methods available for the HostNetworkSystem Managed Object. And low and behold, down the bottom we see the method by which we can modify a portgroup, UpdatePortGroup. We can see that 'networkSystem' is actually a Managed object, but we have to traverse the configManager _Data_ object to get there (because we need to provide the correct context for the managed object reference). Recalling that Get-View can only return _Managed_ Object references, in order to get the HostNetworkSystem Managed Object reference for our particular host, we have to again use Get-View but this time passing it in the networkSystem property of the configManager Data object. Which is a lot easier in PowerShell than it is in English.
$esxNetSys = $esxMoRef.configManager.networkSystemFinally click on the UpdatePortGroup method, and hopefully we're done
$esxNetSysMoRef = Get-View $esxNetSys
OK, finally we see that in order to use the UpdatePortGroup method of a HostNetworkSystem Managed Object, we need to pass in a portgroup name as a string, and a portgroup as a... HostPortGroupSpec object??? WTF!!! Stu, you lead us all this way to a dead fucking end!!! There won't be anything left for Boche to mash at VMworld Europe 2009 by the time I'm done with you!!!
Breathe. Stay Calm. Focus. I have only given you 2 of the 3 tools needed for PowerCLI mastery. Now, we turn to the dark side... err I mean we turn to the SDK documentation, version 2.5 of which is online here.
The online raw SDK documentation is the equivalent of that goo they eat for breakfast in The Matrix - it contains all the things a PowerCLI Master's body needs. You can have a read of the text in the main window there if you want to get a much better explanation of the different object types that what I've given, or you can just jump in, find what you need, then get the fuck out. Yes, while the raw SDK documentation contains everything the body needs, reading it is also a bit like staring at the Sun. Do it for too long, and you'll go insane like that guy in Sunshine.
Open up the online SDK documentation site. Recalling our conversation about the types of object in the VI API, you can reasonably deduce that a 'HostPortGroupSpec' object is likely a Data object. So click on the "Data Object Types" link in the top left hand corner, wait for the list underneath to update then scroll down to find "HostPortGroupSpec", and click on it. This is the equivalent of the part of our PowerShell code where we do:
$pgspec = New-Object VMware.Vim.HostPortGroupSpec
Now where did I pull that “VMware.Vim.” from? We are using a native .net assembly given to us by the VI Toolkit, Vmware.Vim.dll. You probably haven't realised this up until now, I certainly didn’t until about a year ago, but the VI Toolkit is not just PowerShell - it's actually also a native .net wrapper for the VI API, that you can use with C# instead of compiling your own proxy dll from the VI web service. But you don't need to know the nuts and bolts of that - this is something I'm going to ask you to take on faith. You will eat your greens, you will look both ways when crossing the road, and you will know that the VMware.Vim.dll contains a reference for _every_ object available in the underlying API and then some. And for those sceptics out there who don't believe me, execute the following in any old PowerShell window on a box that you have PowerCLI installed on (and make sure u have a large row buffer ;)
$vimDll = [Reflection.Assembly]::LoadWithPartialName("VMware.Vim")
foreach ($type in $vimDll.GetTypes()){write-host $type.name}
Back to the SDK documentation and the code. Now we see what the properties of a HostPortGroupSpec are. There are 4, and all are mandatory. But what’s this, one of those properties is _another_ object, a "HostNetworkPolicy". So what do we need to populate one of those? Click on it and we'll see!
$pgspec.vswitchName = vSwitch0OMG!!! We just opened up a Pandora's box. HostNetworkPolicy has 4 properties, _all of them other objects_! But before you bury your head in your hands, check out those red asterisks - anything with one of those is an optional parameter, which is everything in our case!. Since we're only interested in changing the nic order, which has to do with nic teaming, we're going to leave all properties except the nicTeaming property as empty. Which basically means that this portgroup will inherit the offload policy, security policy and shaping policy of the virtual switch that it is created on. We can see that the nicTeaming property requires a "HostNicTeamingPolicy" object. Click through to that.
$pgspec.Name = "VM Network"
$pgspec.vlanId = "0"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicyOnce again we see a whole bunch of properties, all optional as denoted by the red asterisk. We are interested in changing what vmnics are active, standby and unavailable for this portgroup, so we'll need to populate the nicOrder property. Which as you can tell by now (see, you get the hang of this pretty fast), is another object named 'HostNicOrderPolicy'. So click on through...
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicyFinally! We're at the end of the line. No objects here, just data, and again all optional. As you can see, there are only 2 properties - activeNic and standbyNic. Any nics present on the parent vSwitch but not part of either of these properties is by definition "unused". So in our example, I'll tell you we have 4 vmnics on the parent virtual switch. On the parent virtual switch, all 4 are active (again, this is all in the imaginary pre-existing host this script will be run against). But in this portgroup, we only want vmnic1 and vmnic2 to be active, and vmnic0 and vmnic3 standby. We can see in the SDK documentation that activeNic and standbyNic need values of type 'string array', so we simply do the following to make a valid assignment:
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic1","vmnic2")OK, now we've finally populated our HostPortGroupSpec object with all the required properties, objects, data types and data, it's time to finish off with a call to the UpdatePortGroup() method, via the HostNetworkSystem Managed Object reference we setup in the 3rd line of the script, with the required parameters we got from the MOB. Those being the name of portgroup we want to update, and a HostPortGroupSpec object containing the information we want to apply.
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic3")
$esxNetSysMoRef.UpdatePortGroup($pgspec.Name, $pgspec)Job done.
Now you understand why I'm such a huge fan of PowerCLI. I've had the displeasure of coding against the VI API directly in C#, and Shyam did most of the heavy lifting in Java when we developed statelesx. It often seems convoluted and contradictory, you often find that what you thought would be a simple operation based on another seemingly similar task, is actually anything but. As you've seen in this example, there are layers upon layers upon layers, you can get lost pretty easy. But thanks to the foresight of the VI Toolkit team to expose the underlying VI API via Get-View and provide us with VMware.Vim.dll assembly, anything is possible with the Power of Shanklin. Ooops I mean the Power of Shell ;-). But the Power of Shanklin does compel you.
Hopefully now you can not only do whatever you want to with PowerCLI, but also look at any script that the forum wizards may post, and understand exactly what they’ve done and how you might tweak them to your needs.
Now go forth and conquer. You are Jedi.
PowerCLI Mastery, Volume 2
Thought I was going to drag this out in some scandalous attempt to keep the viewers coming back week after week? Actually, wish I had thought of that before just now. Anyways, let's continue. If you haven't read Volume 1 in the series, go do so now. I'll wait for you.
So continuing on, let’s use the MOB to find a portgroup, and see if that helps us to know what we need to do. The MOB displays the internal object names for things rather than the display name you see in the VI client (the display name is still in there, but it’s usually a property of an object rather than the name of the object itself). If it makes navigation easier, open up the VI Client as well so you can see what internal object names map to which display names. To access the MOB:
1. Browse to https://vchostname/mob
2. Authenticate with whatever credentials you would use for vCenter
You are now for all intents and purposes at the landing page of where 'connect-viserver' takes you.
Click on the "Content" link in the value field of the content property. You can now see the top level of all the available managed objects of the vCenter box you connected to.
We know that networking is configured at the host level, so we need to find a host. The rootFolder property is the start of the inventory, let’s go there.
We are now effectively at the top level "Hosts & Clusters" folder in the VI Client Hosts and Clusters inventory view. As you can see here, in my case there is only 1 Datacenter defined, with an internal object name of 'datacenter-2'. Let’s see what’s in there.
OK, so by looking at the 'name' property, we can see that indeed we are looking at a Datacenter object (name removed to protect the innocent). Pretty easy to guess where we might find hosts... in the hostFolder object! Click through...
Because we have clusters defined in this Datacenter (in this case 2 clusters, 'domain-c28' and 'domain-c297' - again, internal object names), we are now actually looking at the cluster container. Clicking into one of the cluster objects will give us our host list.
OK, here's some useful info. We can see up the top of the screen that we are looking at a "Managed Object" Type. In the VI API there are Managed Objects (higher level objects), Data Objects (lower level objects), and then the data primitives like strings and integers. To illustrate, in this screen the host property contains an array of Managed Objects, the configurationEx property contains a single value that is a Data Object, and the name property contains a single value that is a data primitive. We can also see that the name of this Managed Object Type is ManagedObjectReference:ClusterComputeResource - logical for a cluster. Looking at the properties, we see one named 'host', which is also a Managed Object Type and named 'HostSystem'. Listed in the value field are our 2 host objects. Clicking on one of them, we come to the top level of a HostSystem Managed Object.
Cast your mind back to Volume 1, where I said that 'Get-View' was the window to the underlying objects of the VI API. Now that you know there are 2 different types of objects in the VI API, Managed Objects and Data Objects, I can be more specific and say that Get-View only returns _Managed_ objects. We used Get-View to get a Managed Object (hence why it's common to see variables of this nature have 'mor' or 'moref' in the name) to an ESX host. In order to do so, we had to pass the Get-View cmdlet the Id property of the object returned from Get-VMHost. The reason we had to use the Id property is just the way that the VI Toolkit team designed the Get-View cmdlet to work at this particular level.
And that's where we'll leave it for this installment, Padawans (cue Yoda laugh). In Volume 3 we'll start to disect the script from Volume 1, and follow the rabbit hole a bit deeper.
So continuing on, let’s use the MOB to find a portgroup, and see if that helps us to know what we need to do. The MOB displays the internal object names for things rather than the display name you see in the VI client (the display name is still in there, but it’s usually a property of an object rather than the name of the object itself). If it makes navigation easier, open up the VI Client as well so you can see what internal object names map to which display names. To access the MOB:
1. Browse to https://vchostname/mob
2. Authenticate with whatever credentials you would use for vCenter
You are now for all intents and purposes at the landing page of where 'connect-viserver' takes you.
Click on the "Content" link in the value field of the content property. You can now see the top level of all the available managed objects of the vCenter box you connected to.
We know that networking is configured at the host level, so we need to find a host. The rootFolder property is the start of the inventory, let’s go there.
We are now effectively at the top level "Hosts & Clusters" folder in the VI Client Hosts and Clusters inventory view. As you can see here, in my case there is only 1 Datacenter defined, with an internal object name of 'datacenter-2'. Let’s see what’s in there.
OK, so by looking at the 'name' property, we can see that indeed we are looking at a Datacenter object (name removed to protect the innocent). Pretty easy to guess where we might find hosts... in the hostFolder object! Click through...
Because we have clusters defined in this Datacenter (in this case 2 clusters, 'domain-c28' and 'domain-c297' - again, internal object names), we are now actually looking at the cluster container. Clicking into one of the cluster objects will give us our host list.
OK, here's some useful info. We can see up the top of the screen that we are looking at a "Managed Object" Type. In the VI API there are Managed Objects (higher level objects), Data Objects (lower level objects), and then the data primitives like strings and integers. To illustrate, in this screen the host property contains an array of Managed Objects, the configurationEx property contains a single value that is a Data Object, and the name property contains a single value that is a data primitive. We can also see that the name of this Managed Object Type is ManagedObjectReference:ClusterComputeResource - logical for a cluster. Looking at the properties, we see one named 'host', which is also a Managed Object Type and named 'HostSystem'. Listed in the value field are our 2 host objects. Clicking on one of them, we come to the top level of a HostSystem Managed Object.
Cast your mind back to Volume 1, where I said that 'Get-View' was the window to the underlying objects of the VI API. Now that you know there are 2 different types of objects in the VI API, Managed Objects and Data Objects, I can be more specific and say that Get-View only returns _Managed_ objects. We used Get-View to get a Managed Object (hence why it's common to see variables of this nature have 'mor' or 'moref' in the name) to an ESX host. In order to do so, we had to pass the Get-View cmdlet the Id property of the object returned from Get-VMHost. The reason we had to use the Id property is just the way that the VI Toolkit team designed the Get-View cmdlet to work at this particular level.
$esxMoRef = Get-VMHost esxbox.corp.com | % {Get-View $_.Id}What Managed Object Type did we get a reference to? The exact one we are looking at now - a ManagedObjectReference:HostSystem. And to prove I'm not lying, try this for yourself. Create an $esxHostMoRef variable by issuing the command above in your own environment, and then type $esxHostMoRef.overallStatus and hit enter - it will return whatever value is in the overallStatus row of the webpage in front of you.
And that's where we'll leave it for this installment, Padawans (cue Yoda laugh). In Volume 3 we'll start to disect the script from Volume 1, and follow the rabbit hole a bit deeper.
PowerCLI Mastery, Volume 1
OK, this one has been a looooooooooong time coming. I've been meaning to take a shot at conveying the advanced features of the VI SDK as applied to PowerShell for quite some time now. When Carter corrected me some months ago with regards to what is and isn't possible with PowerCLI (in the first statelesx video, about 1 minute in I mistakenly suggested there was something it _couldn't_ do :-), I've thought I really should get this info out there. Whether I will succeed in this post and the ones that follow is of course for you to decide (this is way to long for a single post), but I'm going to take you through what I think is the fastest way to get your head around the _entire_ SDK. That's right, the whole fucking thing. For as you will see, when you conquer a small piece of the SDK, the rest falls like dominoes.
Now the usual caveats. Yes there are other, possibly more efficient ways to find this info, or write the example code in this article. And some of the things I say will make people who work for Carter cringe. But I've done all this intentionally for the sake of simplicity. And about the the term 'PowerCLI' - Carter dropped it once on Twitter, and all bloggers being opportunists at heart, I'm going to seize on that one with both hands. Besides, it sounds a lot cooler than the VI Toolkit (or at least it more specifically refers to a part of the VI Toolkit). But to avoid any confusion, yes I am talking about this.
It's going to be difficult to go further without the help of an example, so let’s use an example of an operation that there is no native cmdlet for (yet) - modifying the nic order of a portgroup. I don't know how often other people do this, but every ESX blade built where I work gets this treatment. And there are hundreds of those.
Here's the code:
But how did I know those methods and properties are available, what are required and optional values, and what type of values to use? It's not by remote chi, the force, and I didn't pull them out of my arse. I used a combination of Get-View, the MOB and the SDK documentation to do this. No it's not as scary as it sounds, anyone can be on their way to PowerCLI mastery with these tools.
The Get-View cmdlet is part of PowerCLI, and is the window to the underlying VI API. When you need to do something that isn't available via the usual PowerCLI cmdlets, you can use Get-View to acquire an object reference that the underlying API exposes. And of course with PowerShell natively supporting .net objects, you can then access all the associated methods and properties of those underlying objects. That's all well and good, but how do we know what underlying objects, methods and properties we need to use to get our desired result? Continue reading in our next exciting episode, when you will be introduced to the MOB. No, not the sleeps with the fishes type mob, the Managed Object Browser.
Now the usual caveats. Yes there are other, possibly more efficient ways to find this info, or write the example code in this article. And some of the things I say will make people who work for Carter cringe. But I've done all this intentionally for the sake of simplicity. And about the the term 'PowerCLI' - Carter dropped it once on Twitter, and all bloggers being opportunists at heart, I'm going to seize on that one with both hands. Besides, it sounds a lot cooler than the VI Toolkit (or at least it more specifically refers to a part of the VI Toolkit). But to avoid any confusion, yes I am talking about this.
It's going to be difficult to go further without the help of an example, so let’s use an example of an operation that there is no native cmdlet for (yet) - modifying the nic order of a portgroup. I don't know how often other people do this, but every ESX blade built where I work gets this treatment. And there are hundreds of those.
Here's the code:
$esxMoRef = Get-VMHost esxbox.corp.com | % {Get-View $_.Id}
$esxNetSys = $esxMoRef.configManager.networkSystem
$esxNetSysMoRef = Get-View $esxNetSys
$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "VM Network"
$pgspec.vlanId = "0"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic1","vmnic2")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic3")
$esxNetSysMoRef UpdatePortGroup($pgspec.Name,$pgspec)
But how did I know those methods and properties are available, what are required and optional values, and what type of values to use? It's not by remote chi, the force, and I didn't pull them out of my arse. I used a combination of Get-View, the MOB and the SDK documentation to do this. No it's not as scary as it sounds, anyone can be on their way to PowerCLI mastery with these tools.
The Get-View cmdlet is part of PowerCLI, and is the window to the underlying VI API. When you need to do something that isn't available via the usual PowerCLI cmdlets, you can use Get-View to acquire an object reference that the underlying API exposes. And of course with PowerShell natively supporting .net objects, you can then access all the associated methods and properties of those underlying objects. That's all well and good, but how do we know what underlying objects, methods and properties we need to use to get our desired result? Continue reading in our next exciting episode, when you will be introduced to the MOB. No, not the sleeps with the fishes type mob, the Managed Object Browser.
Wednesday 4 February 2009
Interpol Release Travel Advisory for Cannes in Late Feb
That's right, in an uncanny (and unfortunate for VMware) coincidence, Interpol have issued a travel advisory for anyone in the Cote d'Azur region in late February. All travelers and residents are advised to be on the lookout for 2 males apparently involved in an ongoing London-based criminal enterprise. One male is described as 186cm, shaved head, blue eyes, dashing good looks and a strange accent. The other is described as even taller, of Indian descent, and speaking only in the near unintelligible Jafaican slang that is native to East London.
Well that's just bloody typical. Only yesterday did Shyam and I get our travel approval for VMworld Europe 2009, and this happens.
Well that's just bloody typical. Only yesterday did Shyam and I get our travel approval for VMworld Europe 2009, and this happens.
Only 1 Week Left for vIdiot Nominations!
The vExpert nomination process may be wrapping up in a few days, but you've still got a little over 1 week to get those vIdiot nominations in! We've had some good ones so far, this will be a difficult one to judge. Currently I'm leaning towards some cracking entries from the VMware forums submitted by Rodos (and no they're not innocent questions from newbies - they're ppl claiming to know better about things that they clearly do not), and someone else has put in a great entry from a Google Groups thread! Not to mention Chris Wolf's inadvertent nomination via an anonymous commenter on his blog for use of the word 'Irregardless'... that's not a word Chris, but oh too late, what a world, what a world!
So search high and low ppl, scan the corners of the interwebs for those vIdiots, and out them! Send entries to vinternals at gmail dot com.
So search high and low ppl, scan the corners of the interwebs for those vIdiots, and out them! Send entries to vinternals at gmail dot com.
Saturday 31 January 2009
Citrix Marketing Bullsh_t Dispelled by VMware (again)
I can't tell you how pleased I was to see VMware _finally_ publish some data on a more realistic XenApp scenario than they have previously. Back in the days they used to push a single vCPU for Terminal Servers argument, which as any Citrix guy will know is sub-optimal in more than a few aspects (mostly relating to the relationship of the TS console session and logons to CPU0).
Now I'll be the first to admit that any kind of test and resulting statistical analysis can be very subjective, but I'll tell you what I think is the most important point to take from this latest study from VMware - the performance isn't that different. Sure ESX came out on top, but not by much (I'm sure Citrix will come out with some kind of data that shows the contrary before too long, and that's fine). And why is that important? Because I am sick and tired of the marketing bullshit from Citrix and subsequent throwaway one-liners that propagate through the intertubes and into the minds of Administrators in the enterprise, regarding XenServer being "optimised for XenApp". I have _never_ seen any further details on what this actually means.
Well I'm throwing down the gauntlet on this one. I'm opening up the comments on this post, in the hope that Simon Crosby, Ian Pratt, or anyone else I highly respect on a technical level who works for Citrix to tell us all what the fuck these XenApp specific optimisations in XenServer actually are and why the same configuration could not be implemented with tweaks to ESX (not that they're even necessary, going by the VMware post). I'll even take a reply from hype-master Roger Klorese. If it is some kind of secret, I have no problem with that - just tell us so (and I'll continue to call it complete BS). If it's something that can only be disclosed under NDA, then tell us that too because I guarantee you that 90% of the people who read this blog regularly are working for large enterprises who have NDA's in place with Citrix, so we can follow that avenue if necessary (and of course honour the NDA - I'm not stupid enough to lose my job for the sake of this blog). Cut the bullshit Citrix - we just want to know.
UPDATE As numerous commenters have pointed out, I completely neglected to mention the excellent work of Project VRC. But again, in line with my original comment, the differences in hypervisors are so minor that it really doesn't matter - as Duncan said, the main thing is that Citrix workloads are finally viable targets for virtualisation, because the difference to physical is also insignificant when you consider all the benefits of virtualisation.
As for Simon Crosby's retort, I won't bother picking through it. Citrix are easily as guilty as anyone when it comes to spreading one sided "studies" under the guise of science, and their best buddies at Microsoft are the kings of the hill in that arena. Here's a tip for you Simon - Microsoft are also the kings of "embrace, extend, extinguish", remember? We all know what's gonna happen when they enter the connection broker market.
BUT I will say this much in credit to Simon - he hit the nail on the head with his comments regarding VMware's draconian "thou shalt not publish benchmarks" stance. This has been a bugbear of mine for a long time, and I've said as much to fairly senior people at VMware. A change in that would be most welcome indeed.
Also as a few of the anonymous commenters pointed out, Citrix offered this explanation 6 months ago as to what the 'secret XenApp optimisation sauce' is. Which I can only assume is now completely null and void with the widespread availability of RVI / EPT. Oh well, was fun while it lasted I guess.
Now I'll be the first to admit that any kind of test and resulting statistical analysis can be very subjective, but I'll tell you what I think is the most important point to take from this latest study from VMware - the performance isn't that different. Sure ESX came out on top, but not by much (I'm sure Citrix will come out with some kind of data that shows the contrary before too long, and that's fine). And why is that important? Because I am sick and tired of the marketing bullshit from Citrix and subsequent throwaway one-liners that propagate through the intertubes and into the minds of Administrators in the enterprise, regarding XenServer being "optimised for XenApp". I have _never_ seen any further details on what this actually means.
Well I'm throwing down the gauntlet on this one. I'm opening up the comments on this post, in the hope that Simon Crosby, Ian Pratt, or anyone else I highly respect on a technical level who works for Citrix to tell us all what the fuck these XenApp specific optimisations in XenServer actually are and why the same configuration could not be implemented with tweaks to ESX (not that they're even necessary, going by the VMware post). I'll even take a reply from hype-master Roger Klorese. If it is some kind of secret, I have no problem with that - just tell us so (and I'll continue to call it complete BS). If it's something that can only be disclosed under NDA, then tell us that too because I guarantee you that 90% of the people who read this blog regularly are working for large enterprises who have NDA's in place with Citrix, so we can follow that avenue if necessary (and of course honour the NDA - I'm not stupid enough to lose my job for the sake of this blog). Cut the bullshit Citrix - we just want to know.
UPDATE As numerous commenters have pointed out, I completely neglected to mention the excellent work of Project VRC. But again, in line with my original comment, the differences in hypervisors are so minor that it really doesn't matter - as Duncan said, the main thing is that Citrix workloads are finally viable targets for virtualisation, because the difference to physical is also insignificant when you consider all the benefits of virtualisation.
As for Simon Crosby's retort, I won't bother picking through it. Citrix are easily as guilty as anyone when it comes to spreading one sided "studies" under the guise of science, and their best buddies at Microsoft are the kings of the hill in that arena. Here's a tip for you Simon - Microsoft are also the kings of "embrace, extend, extinguish", remember? We all know what's gonna happen when they enter the connection broker market.
BUT I will say this much in credit to Simon - he hit the nail on the head with his comments regarding VMware's draconian "thou shalt not publish benchmarks" stance. This has been a bugbear of mine for a long time, and I've said as much to fairly senior people at VMware. A change in that would be most welcome indeed.
Also as a few of the anonymous commenters pointed out, Citrix offered this explanation 6 months ago as to what the 'secret XenApp optimisation sauce' is. Which I can only assume is now completely null and void with the widespread availability of RVI / EPT. Oh well, was fun while it lasted I guess.
Friday 30 January 2009
Next London VMUG - March 10th
As announced by Al Davies and reported by others, the next London VMware User Group meet will be on March 10th.
Come along to hopefully learn something, and say hi to some of the big names in the Virtual blogosphere such as Mike "RTFM or STFU" Laverick, Alan "Oui Oui, Poo Poo" Renouf, and the lads from Xtravirt. There's always the chance of Tom Howarth dropping by, as well as any number of people from VMware UK. The "bloggers attempting to get fame by association" from vinternals, Shyam and I, will also be there.
And if none of that interests you, then the pub session afterwards should. Pint of Peroni, thanks!
Come along to hopefully learn something, and say hi to some of the big names in the Virtual blogosphere such as Mike "RTFM or STFU" Laverick, Alan "Oui Oui, Poo Poo" Renouf, and the lads from Xtravirt. There's always the chance of Tom Howarth dropping by, as well as any number of people from VMware UK. The "bloggers attempting to get fame by association" from vinternals, Shyam and I, will also be there.
And if none of that interests you, then the pub session afterwards should. Pint of Peroni, thanks!
Wednesday 28 January 2009
Shanklin Elevated From "Demigod" to "FullGod"... Film at 11.
After having a look around the new VI Toolkit release today, and the accompanying videos and documentation, I am convinced Carter will be hailed as a new God by VI admins the world over. I mean, even Boche has stated this release may be enough to move his lazy ass into finally learning the ways of the 'Shell (I've been at him for ages about this, but alas he just ignores me).
I sooooo want to show the VI Java API some love (which was quietly updated to 1.0U1 only days ago), but with Windows Server 2008 R2 Core finally providing .net support, I'm more likely to wrap PowerShell scripts up in webservices for interop purposes than get dirty with Java (and that's no reflection on the quality of what Steve Jin has done, which is top class).
Massive props to Carter and his team. With tools like this, I don't give a fuck if vCenter or anything else becomes Linux based. Lets hope vCenter Orchestrator has native PowerShell support.
I sooooo want to show the VI Java API some love (which was quietly updated to 1.0U1 only days ago), but with Windows Server 2008 R2 Core finally providing .net support, I'm more likely to wrap PowerShell scripts up in webservices for interop purposes than get dirty with Java (and that's no reflection on the quality of what Steve Jin has done, which is top class).
Massive props to Carter and his team. With tools like this, I don't give a fuck if vCenter or anything else becomes Linux based. Lets hope vCenter Orchestrator has native PowerShell support.
VI Toolkit 1.5 Released
The long awaited version 1.5 of the VI Toolkit has finally GA'ed, as Carter announced in the wee hours of this morning (2:30AM... I always wondered if Carter was a robot sent from the future, hopefuly not to destroy us all though).
Anyway 'nuff said, I'm gonna go get it now!
Anyway 'nuff said, I'm gonna go get it now!
Sunday 18 January 2009
The Myth of Infrastructure Contention
Back... caught u lookin for the same thing. It's a new thing, check out this... oh no wait. It ain't a new thing, it's just another Sunday Arvo Architecture And Philoshopy post. This time I'm going to focus on a long time thorn in many of our sides - the myth of infrastructure resource contention.
This ugly beast rears it's head in many ways, but in particular when consolidating workloads or changing hardware standards (such as standardising on blade). Of course, the people raising these arguments are often server admins with little or no knowledge of the storage and network architecture in their environments, or consultants who have either never worked in large environments or also do not know the storage and network architectures of the environment they have come into. Which is not their fault - due to the necessary delineation of responsibility in any enterprise, they just don't get exposure to the big picture. And again, I should say from the outset that I'm talking ENTERPRISE people! Seriously, if I cop shit from one more person who claims to know better based on their 20 "strong" ESX infrastructure or home fucking lab, I am going to break out the shuriken. YES THE ENTERPRISE IS DIFFERENT. If you have never worked in a large environment, you can probably stop reading right now (unless you want to work in a large environment, in which case you should pay close attention). Can you tell how much these baseless concerns get to me? Now where was I...
OK a few more disclaimers. In this post I will try to stay as generic as possible to allow for broader applicability, and focus on single paths and simplistic components for the sake of clarity. Yes I know about Layer 3 switches and other convergent network devices and topologies, but they don't help to clarify things for those who may not know of such things. Additionally, the diagrams below are a wierd kind of mish-mash of various things I've seen in my time in several large enterprises, and I suck at Visio. Again, I have labelled things for clarity more than accuracy, and chopped stuff down in the name of broad applicability. Keep that in mind before you write to me saying I've got it all wrong.
IP Networks
So lets tackle the big one first, IP networks. Before virtualisation, your network may have looked something like this:
Does that surprise you? If it does, go ask one of your network team to draw out what a typical server class network looks like, from border to server. I bet I'm not far off. Go and do it now, I'll wait for you to get back.
OK, enlightened now? And in fact if you are, the penny has probably already dropped. But in case it hasn't, lets see what happens when the virtualisatoin train comes rolling in, and your friendly architecture and engineering team propose putting those 100 phsyicals into a blade chassis. It is precisely at this point that most operations staff, without a view of the big picture, start screaming bloody murder. "You idiot designers, how the hell do you think you can connect 100 boxes with only 4GB (active) links!!! @$%#@%# no way I'm letting that into production you @#%$%!!!". However, when we virtualise those 100 physical boxes and throw them all into a blade chassis, our diagram becomes:
OK, _now_ the penny has definitely dropped (or you shouldn't have administrative access to production systems). IT DOESN"T MATTER WHAT IS BELOW THE ACCESS LAYER. Because a single hop away (or 2 if you're lucky), all that bandwidth is concentrated by an order of magnitude. The networks guys have known this all along. They probably laughed at the server guys demands for GbE to the endpoints, knowing that in the grand scheme of things it would make fuck all difference in 90% of cases. But they humoured us anyway. And lucky for them they did, because the average network guy's mantra of "the core needs to be a multiple of the edge" needs to be tossed out on it's arse, for different reasons. But that's another post :-).
Fibre Channel Networks
I know I know, I really don't need to be as blatant about it this time, because you know I'm going to follow the exact same logic with storage. But just to drive the point home, here again we have our before virtualisation infrastructure:
And again, after sticking everything onto a blade chassis:
I don't think the above needs any futher explanation.
I'm sure there are a million variations out there that may give rise to what some may think as legitimate arguments. You may have a dedicated backup network, it may even be non-routed. To which I would ask, what is the backup server connected at? What are you backing up to? Whats the overall throughput of that backup system? Point is, there will _always_ be concentration of bandwidth on the backend, be it networking or storage, and your physical boxes don't use anywhere near the amount of bandwdith that you think they do. You may get the odd outlier, sure. Just stick it on it's own box, but still put ESX underneath it - even without the added benefits of SAN and cluster membership, from an administrative perspective you still get many advantages of virtualising the OS (remember, enterprise. We don't pay on a per-host basis, so the additional cost of ESX doesn't factor in for enterprises like it would for smaller shops).
OK, time to wrap this one up. Your environment may vary from the diagrams above, but you _will_ have concentration points like those above, somewhere. That being the case, if you don't have network or storage bandwidth problems before virtualisation, don't think that you will have them afterwards just because you massively cut the aggregate endpoint connectivity.
This ugly beast rears it's head in many ways, but in particular when consolidating workloads or changing hardware standards (such as standardising on blade). Of course, the people raising these arguments are often server admins with little or no knowledge of the storage and network architecture in their environments, or consultants who have either never worked in large environments or also do not know the storage and network architectures of the environment they have come into. Which is not their fault - due to the necessary delineation of responsibility in any enterprise, they just don't get exposure to the big picture. And again, I should say from the outset that I'm talking ENTERPRISE people! Seriously, if I cop shit from one more person who claims to know better based on their 20 "strong" ESX infrastructure or home fucking lab, I am going to break out the shuriken. YES THE ENTERPRISE IS DIFFERENT. If you have never worked in a large environment, you can probably stop reading right now (unless you want to work in a large environment, in which case you should pay close attention). Can you tell how much these baseless concerns get to me? Now where was I...
OK a few more disclaimers. In this post I will try to stay as generic as possible to allow for broader applicability, and focus on single paths and simplistic components for the sake of clarity. Yes I know about Layer 3 switches and other convergent network devices and topologies, but they don't help to clarify things for those who may not know of such things. Additionally, the diagrams below are a wierd kind of mish-mash of various things I've seen in my time in several large enterprises, and I suck at Visio. Again, I have labelled things for clarity more than accuracy, and chopped stuff down in the name of broad applicability. Keep that in mind before you write to me saying I've got it all wrong.
IP Networks
So lets tackle the big one first, IP networks. Before virtualisation, your network may have looked something like this:
Does that surprise you? If it does, go ask one of your network team to draw out what a typical server class network looks like, from border to server. I bet I'm not far off. Go and do it now, I'll wait for you to get back.
OK, enlightened now? And in fact if you are, the penny has probably already dropped. But in case it hasn't, lets see what happens when the virtualisatoin train comes rolling in, and your friendly architecture and engineering team propose putting those 100 phsyicals into a blade chassis. It is precisely at this point that most operations staff, without a view of the big picture, start screaming bloody murder. "You idiot designers, how the hell do you think you can connect 100 boxes with only 4GB (active) links!!! @$%#@%# no way I'm letting that into production you @#%$%!!!". However, when we virtualise those 100 physical boxes and throw them all into a blade chassis, our diagram becomes:
OK, _now_ the penny has definitely dropped (or you shouldn't have administrative access to production systems). IT DOESN"T MATTER WHAT IS BELOW THE ACCESS LAYER. Because a single hop away (or 2 if you're lucky), all that bandwidth is concentrated by an order of magnitude. The networks guys have known this all along. They probably laughed at the server guys demands for GbE to the endpoints, knowing that in the grand scheme of things it would make fuck all difference in 90% of cases. But they humoured us anyway. And lucky for them they did, because the average network guy's mantra of "the core needs to be a multiple of the edge" needs to be tossed out on it's arse, for different reasons. But that's another post :-).
Fibre Channel Networks
I know I know, I really don't need to be as blatant about it this time, because you know I'm going to follow the exact same logic with storage. But just to drive the point home, here again we have our before virtualisation infrastructure:
And again, after sticking everything onto a blade chassis:
I don't think the above needs any futher explanation.
I'm sure there are a million variations out there that may give rise to what some may think as legitimate arguments. You may have a dedicated backup network, it may even be non-routed. To which I would ask, what is the backup server connected at? What are you backing up to? Whats the overall throughput of that backup system? Point is, there will _always_ be concentration of bandwidth on the backend, be it networking or storage, and your physical boxes don't use anywhere near the amount of bandwdith that you think they do. You may get the odd outlier, sure. Just stick it on it's own box, but still put ESX underneath it - even without the added benefits of SAN and cluster membership, from an administrative perspective you still get many advantages of virtualising the OS (remember, enterprise. We don't pay on a per-host basis, so the additional cost of ESX doesn't factor in for enterprises like it would for smaller shops).
OK, time to wrap this one up. Your environment may vary from the diagrams above, but you _will_ have concentration points like those above, somewhere. That being the case, if you don't have network or storage bandwidth problems before virtualisation, don't think that you will have them afterwards just because you massively cut the aggregate endpoint connectivity.
Thursday 15 January 2009
vInternals Launches vIdiot Program!
Funny thing, nature. As the Tao te Ching says, "when beauty is abstracted, then ugliness has been implied." That is, for something to be considered "beautiful", there must be something else considered "ugly". And so too, there cannot be a "vExpert" without a "vIdiot".
So we here at vinternals have taken it upon ourselves once again to be the Yin to VMware's Yang, and commenced nominations for the inaugural vIdiot award. The nomination process itself is pretty straight forward. Simply find a dumbarse quote from someone who is among the un-enlightened, and forward it along with a link to the source. Here are some examples:
I probably burned a few bridges just now. Oh well, so I'll never be an MVP, sif care. But a vExpert I could be, with some help from you! So if you value a balanced view, mixed humour and deep tech, and thoughts on the future just as much as the pure technical info that appears more regularly on other sites (like those listed in the "linkage" section down the right there) get on over to the nomination page and do me proud. Anyone who mails me on vinternals at gmail dot com can find out my name. Shameless? Don't know the meaning of the word ;)
Send your vIdiot entries to vinternals at gmail dot com, entries will close on February 13th and results posted around the end of Feb!
UPDATE Woops, looks like a few of us got it wrong with the nomination process. So I've struck out the shameless plea for nominations as I've already had at least one!
So we here at vinternals have taken it upon ourselves once again to be the Yin to VMware's Yang, and commenced nominations for the inaugural vIdiot award. The nomination process itself is pretty straight forward. Simply find a dumbarse quote from someone who is among the un-enlightened, and forward it along with a link to the source. Here are some examples:
Can anyone tell me how VMware keep saying that they are using Para metal Virtualization and overcommiting at the same time?!!! - Mohamed Fawzi
In the case of unplanned downtime, VMotion can’t live migrate because there is no warning. - Jeff Woolsey
We’re designing Windows Server virtualization to scale up to 64 processors, which I’m proud to say is something no other vendor’s product supports. We are also providing a much more dynamic VM environment with hot-add of processors, memory, disk and networking as well a greater scalability with more SMP support and memory. - Mike Neil (talking up what will be in Hyper-V 1.0 one month before they chopped everything)
We know you don't get it. That's fine. Chin up, and do please keep writing. - Simon Crosby (directed at Mike D)
You’ve got to question whether it’s worth paying $50,000 for that. I know the VMware camp go on about features like VMotion, but for $50,000 I could pay someone to move my virtual machines for me. - David Furey
I probably burned a few bridges just now. Oh well, so I'll never be an MVP, sif care. But a vExpert I could be, with some help from you! So if you value a balanced view, mixed humour and deep tech, and thoughts on the future just as much as the pure technical info that appears more regularly on other sites (like those listed in the "linkage" section down the right there) get on over to the nomination page and do me proud. Anyone who mails me on vinternals at gmail dot com can find out my name. Shameless? Don't know the meaning of the word ;)
Send your vIdiot entries to vinternals at gmail dot com, entries will close on February 13th and results posted around the end of Feb!
UPDATE Woops, looks like a few of us got it wrong with the nomination process. So I've struck out the shameless plea for nominations as I've already had at least one!
Sunday 11 January 2009
Boot Windows 7 From VHD!
In the virtualisation space, one of the often demo'ed features of the upcoming Windows 7 / 2008 R2 is the ability to boot directly from VHD. Microsoft have effectively created a "loopback HBA", which the bootloader can use to address VHD's just like a regular disk. This is pretty cool for a whole host of reasons, and easy to achieve.
1. If you're going to install another Windows 7, first set the boot menu description of your current installation with bcdedit /set {current} description "Windows 7 HDD"
2. Now create a new VHD. You can do this via the GUI in disk management, or via the CLI with diskpart. Since you're gonna need diskpart for the install, might as well use that now. I'm sure you can figure out what I'm doing with diskpart create vdisk file=d:\vhd\windows7.vhd type=fixed maximum=20000
3. Installation time. Drop in your Windows 7 installation media, reboot and when u get to the first screen prompt of the installation hit Shift+F10 to bring up a command window. Run diskpart, then enter select vdisk file=d:\vhd\windows7.vhd, then enter attach vdisk, then exit diskpart. Note that if you have multiple physical disks, Windows PE may not have honoured your drive letter assignments. Enter list vol from within diskpart to see what it's done if you run into any trouble.
4. Continue the installation process, and when you get to the choice of disks to install onto, you should see your VHD sitting in the list of available options just like a regular disk.
5. Select the VHD and go read something while the install finishes. You're done!
When the machine reboots, you'll notice that the VHD boot menu option is now the default. This can be easily changed using bcdedit from within either of the Windows environments you have booted into.
Next thing I'll probably try is to see just how much I can cut down the original install so I can use VHD's for everything, kind of like a semi-client hypervisor. Sounds like a perfect job for Server Core... hopefully it's functional enough to be able to do this, I can't imagine why it wouldn't be.
UPDATE Just been messing around with server core in a VM, looks like all systems are go. Using the same commands as above I could create / attach / format a VHD. Server core looks _great_ in 2008 R2. Installing PowerShell and launching it nearly brought a tear to my eye... finally server core has arrived.
UPDATE 2 Reader Patrick S has just pinged me to let me know that this technique also allows for installation of Windows XP / Server 2003 as well! I assume you'd need to use the Windows 7 PE environment or something... but he's done it, so there's a way. Nice one Patrick, thanks for the heads up!
1. If you're going to install another Windows 7, first set the boot menu description of your current installation with bcdedit /set {current} description "Windows 7 HDD"
2. Now create a new VHD. You can do this via the GUI in disk management, or via the CLI with diskpart. Since you're gonna need diskpart for the install, might as well use that now. I'm sure you can figure out what I'm doing with diskpart create vdisk file=d:\vhd\windows7.vhd type=fixed maximum=20000
3. Installation time. Drop in your Windows 7 installation media, reboot and when u get to the first screen prompt of the installation hit Shift+F10 to bring up a command window. Run diskpart, then enter select vdisk file=d:\vhd\windows7.vhd, then enter attach vdisk, then exit diskpart. Note that if you have multiple physical disks, Windows PE may not have honoured your drive letter assignments. Enter list vol from within diskpart to see what it's done if you run into any trouble.
4. Continue the installation process, and when you get to the choice of disks to install onto, you should see your VHD sitting in the list of available options just like a regular disk.
5. Select the VHD and go read something while the install finishes. You're done!
When the machine reboots, you'll notice that the VHD boot menu option is now the default. This can be easily changed using bcdedit from within either of the Windows environments you have booted into.
Next thing I'll probably try is to see just how much I can cut down the original install so I can use VHD's for everything, kind of like a semi-client hypervisor. Sounds like a perfect job for Server Core... hopefully it's functional enough to be able to do this, I can't imagine why it wouldn't be.
UPDATE Just been messing around with server core in a VM, looks like all systems are go. Using the same commands as above I could create / attach / format a VHD. Server core looks _great_ in 2008 R2. Installing PowerShell and launching it nearly brought a tear to my eye... finally server core has arrived.
UPDATE 2 Reader Patrick S has just pinged me to let me know that this technique also allows for installation of Windows XP / Server 2003 as well! I assume you'd need to use the Windows 7 PE environment or something... but he's done it, so there's a way. Nice one Patrick, thanks for the heads up!
Wednesday 7 January 2009
VMworld Europe 2009 - Boche Will Be There, Will You?
Hot in off the intertubes - bench pressing maniac Jason Boche, who also has a popular virtualisation blog (that's right people, it's spelled with an 's'. Sometimes I wonder where the "English" in "US English" went) has announced he will be attending VMware Europe 2009. This is great news, because it's more likely that my as yet unannounced 'US vs EU Bloggers Sumo Challenge' will go ahead if more US bloggers actually get to the event :)
Oh, those lowly bloggers from vinternals will be there as well, but who cares.
UPDATE Legendary Scott Herold will also be there to represent Team USA! Thank god Oglesby doesn't blog!
Oh, those lowly bloggers from vinternals will be there as well, but who cares.
UPDATE Legendary Scott Herold will also be there to represent Team USA! Thank god Oglesby doesn't blog!
VMware 2009 Product Lineup
I was under the impression that some of these products were still in super secret squirrel status, but as they're now appearing on the VMware Products homepage, I guess that's no longer the case!
Since you're all going to go and check out the VMware spiel anyways, I won't bother repeating it here.
What I will say though is that I'm not sure how wise it is for VMware to simultaneously spread themselves even thinner by releasing more products, and eat into the opportunities for ISV's. 2009 is going to be a tough year for everyone, and a healthy ISV market is good for a software company. The last thing VMware should be doing is try and offer everything themselves, thus driving ISV's like Veeam, Quest and the many others into Microsoft's camp where they will no doubt be welcomed with open arms!
Since you're all going to go and check out the VMware spiel anyways, I won't bother repeating it here.
What I will say though is that I'm not sure how wise it is for VMware to simultaneously spread themselves even thinner by releasing more products, and eat into the opportunities for ISV's. 2009 is going to be a tough year for everyone, and a healthy ISV market is good for a software company. The last thing VMware should be doing is try and offer everything themselves, thus driving ISV's like Veeam, Quest and the many others into Microsoft's camp where they will no doubt be welcomed with open arms!
Sunday 4 January 2009
The Rise of the Stupid Endpoint
Welcome to the first SAAAP of 2009. And boy am I feeling philosophical today :-). The similarities between this post and David Isenberg's article Rise of the Stupid Network end with the title, but if you haven't read Isenberg's article you should do so - the principles he outlined over 10 years ago are just as applicable today to network infrastructure.
The Failure of Distributed Computing
The proliferation of distributed computing in the datacenter was essentially driven by the low cost of x86 hardware. Rather than spending huge sums on mainframe or proprietary Unix infrastructure and have it sit there underutilised for a long period (hmmm, sound familiar?) or use it for non-critical applications, why not buy smaller, cheaper x86 servers instead? The managment of these many small systems was never really factored into the equation however, and before too long there were more applications for management of this infrastructure than you could poke a stick at. But for a long time, those management systems were generally lacking in intelligence. And because of that, more staff were needed to manage the management systems, respond to alerts etc etc. x86 didn't turn out to be as cheap as promised, not because of the hardware but because of the lack of intelligent management systems.
What We Learned from Management Systems
The distributed computing management systems taught a new generation of system administrators (as was I) what had existed in the mainframe infrastructure for decades - the advantages of centralisation. Monitoring baselines, patch and software distribution, backup, job scheduling... these things have always been centralised in enterprise distributed computing environments. Such environments would be extremely expensive to operate otherwise. Yet for some reason this train of thought never really made it all the way to the personality of the endpoint. Because of this, system level backup and restore is of paramount importance in todays datacenter. DR and lifecycle management is a painfully manual and labour intensive process. Administrative overhead is exacerbated.
Endpoint Stupidity, Centralised Intelligence
You cannot make something stupid without simultaneously making something else intelligent. And that intelligence needs to be centralised. A single endpoint has many touch points in the enterprise. Are all these touch points still required? Is the cost of removing them greater than the cost of living with them? Is the pain simply due to a lack of intelligence in the tools, or the processes that you follow? Orchestration tools may be the solution for this, but do not fall into the trap of merely alleviating the pain and not addressing the underlying cause.
Infrastructure Like Water
"The height of cultivation is really nothing special. It is merely simplicity; the ability to express the utmost with the minimum." - Bruce Lee
That's what it all boils down to. In order for distributed computing infrastructure to work, we need to simplify the endpoint. When the endpoint is simple, it can be formless like a cloud (sorry, I couldn't resist), empty like a cup. You cannot have elasticity in the datacenter without this simplification. A piece of hardware may be running ESX today, Windows tomorrow, and shutdown the next day. In order to do that, the personality of the hardware cannot exist on the hardware itself. Likewise with a workload. It may ask for one level of resources today and another tomorrow, it doesn't matter. It may be running in one location today and another tomorrow, it shouldn't matter. Virtualisation has obviously made this much easier than it would have been otherwise, and I'm taking virtualisation as whole here - not just hypervisors, but VLANs, VSANs, VIPs, DDNS etc etc. But the same principles need to apply to both physical and virtual infrastructure.
End Game
So that's the what, it's up to people like you and me to figure out the how. With problems like these to solve, how can 2009 not be a good year :-)
The Failure of Distributed Computing
The proliferation of distributed computing in the datacenter was essentially driven by the low cost of x86 hardware. Rather than spending huge sums on mainframe or proprietary Unix infrastructure and have it sit there underutilised for a long period (hmmm, sound familiar?) or use it for non-critical applications, why not buy smaller, cheaper x86 servers instead? The managment of these many small systems was never really factored into the equation however, and before too long there were more applications for management of this infrastructure than you could poke a stick at. But for a long time, those management systems were generally lacking in intelligence. And because of that, more staff were needed to manage the management systems, respond to alerts etc etc. x86 didn't turn out to be as cheap as promised, not because of the hardware but because of the lack of intelligent management systems.
What We Learned from Management Systems
The distributed computing management systems taught a new generation of system administrators (as was I) what had existed in the mainframe infrastructure for decades - the advantages of centralisation. Monitoring baselines, patch and software distribution, backup, job scheduling... these things have always been centralised in enterprise distributed computing environments. Such environments would be extremely expensive to operate otherwise. Yet for some reason this train of thought never really made it all the way to the personality of the endpoint. Because of this, system level backup and restore is of paramount importance in todays datacenter. DR and lifecycle management is a painfully manual and labour intensive process. Administrative overhead is exacerbated.
Endpoint Stupidity, Centralised Intelligence
You cannot make something stupid without simultaneously making something else intelligent. And that intelligence needs to be centralised. A single endpoint has many touch points in the enterprise. Are all these touch points still required? Is the cost of removing them greater than the cost of living with them? Is the pain simply due to a lack of intelligence in the tools, or the processes that you follow? Orchestration tools may be the solution for this, but do not fall into the trap of merely alleviating the pain and not addressing the underlying cause.
Infrastructure Like Water
"The height of cultivation is really nothing special. It is merely simplicity; the ability to express the utmost with the minimum." - Bruce Lee
That's what it all boils down to. In order for distributed computing infrastructure to work, we need to simplify the endpoint. When the endpoint is simple, it can be formless like a cloud (sorry, I couldn't resist), empty like a cup. You cannot have elasticity in the datacenter without this simplification. A piece of hardware may be running ESX today, Windows tomorrow, and shutdown the next day. In order to do that, the personality of the hardware cannot exist on the hardware itself. Likewise with a workload. It may ask for one level of resources today and another tomorrow, it doesn't matter. It may be running in one location today and another tomorrow, it shouldn't matter. Virtualisation has obviously made this much easier than it would have been otherwise, and I'm taking virtualisation as whole here - not just hypervisors, but VLANs, VSANs, VIPs, DDNS etc etc. But the same principles need to apply to both physical and virtual infrastructure.
End Game
So that's the what, it's up to people like you and me to figure out the how. With problems like these to solve, how can 2009 not be a good year :-)
Friday 2 January 2009
VI Toolkit and the PowerShell 2.0 Integrated Script Editor
Finally got a chance to install the PowerShell 2.0 CTP3, and have a look at the Integrated Script Editor (ISE). It has all the basic functionality you would expect from a scripting editor, including a very handy tab completion of cmdlets. It didn't pick up the VI Toolkit cmdlets straight away for me though, so here's how you'd enable that if you run into the same issue.
The ISE only picks up snapins that are loaded via one of the 'AllHosts' profiles, which is either '$pshome\profile.ps1' (the AllUsers\AllHosts profile) or '$home\[My] Documents\WindowsPowerShell\profile.ps1' (the CurrentUser\AllHosts profile). Until now I have stuck with using the CurrentUser\CurrentHost profile to customise my shell, which is in '$home\[My] Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'. As a result, all the VI Toolkit cmdlets are available in any powershell session, however they aren't available in the ISE.
So the fix was obviously simple - rename '$home\[My] Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1' to 'profile.ps1'. For anyone out there who doesn't load the VI Toolkit with their normal powershell environment, simply add the following line...
add-PSSnapin VMware.VimAutomation.Core
... to $home\[My] Documents\WindowsPowerShell\profile.ps1 and you're done!
The ISE only picks up snapins that are loaded via one of the 'AllHosts' profiles, which is either '$pshome\profile.ps1' (the AllUsers\AllHosts profile) or '$home\[My] Documents\WindowsPowerShell\profile.ps1' (the CurrentUser\AllHosts profile). Until now I have stuck with using the CurrentUser\CurrentHost profile to customise my shell, which is in '$home\[My] Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'. As a result, all the VI Toolkit cmdlets are available in any powershell session, however they aren't available in the ISE.
So the fix was obviously simple - rename '$home\[My] Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1' to 'profile.ps1'. For anyone out there who doesn't load the VI Toolkit with their normal powershell environment, simply add the following line...
add-PSSnapin VMware.VimAutomation.Core
... to $home\[My] Documents\WindowsPowerShell\profile.ps1 and you're done!
Thursday 1 January 2009
Deploying OVF's with the VMware OVF Tool
I finally got around to OVF'ing a Windows XP build. Although I have an unattended setup XP .iso which makes building XP VM's hassle free and pretty fast, it's obviously nowhere near as hassle free and fast as deploying from OVF!
Although you can import OVF's with VMware Workstation, it's a bit cumbersome. I've also had some wierd experiences with it, such as the "split disk into 2GB files" option being ticked and greyed out when I didn't want the option selected (both for export and import operations). VMware Player is no better, as you can't control where the OVF is deployed to. Back when we were developing statelesx, we used a Java based version of the OVF Tool from VMware to create the virtual appliance package. But someone at VMware has been quietly beavering away (I say quietly because I haven't seen _any_ publicity from VMware about this tool), and it's now at a 1.0.0 build (although still labelled as a "technology preview").
And just like Boche, they've injected some steroids into the tool... for example (note I haven't tried this... yet) you could pull an OVF straight off a webserver and plonk it onto a host, specifying datastore and portgroup details with:
ovfTool "http://my_ovf_server/ovfs/myAppliance.ovf" "vi://username:pass@my_vc_server/my_datacenter/host/my_host&datastore=my_datastore&net:ovf_network_name=vc_network_name"
So check it out, it's a nice and fast way to deploy OVF's locally for use with Workstation or Player, but also has some serious enterprise functionality.
Although you can import OVF's with VMware Workstation, it's a bit cumbersome. I've also had some wierd experiences with it, such as the "split disk into 2GB files" option being ticked and greyed out when I didn't want the option selected (both for export and import operations). VMware Player is no better, as you can't control where the OVF is deployed to. Back when we were developing statelesx, we used a Java based version of the OVF Tool from VMware to create the virtual appliance package. But someone at VMware has been quietly beavering away (I say quietly because I haven't seen _any_ publicity from VMware about this tool), and it's now at a 1.0.0 build (although still labelled as a "technology preview").
And just like Boche, they've injected some steroids into the tool... for example (note I haven't tried this... yet) you could pull an OVF straight off a webserver and plonk it onto a host, specifying datastore and portgroup details with:
ovfTool "http://my_ovf_server/ovfs/myAppliance.ovf" "vi://username:pass@my_vc_server/my_datacenter/host/my_host&datastore=my_datastore&net:ovf_network_name=vc_network_name"
So check it out, it's a nice and fast way to deploy OVF's locally for use with Workstation or Player, but also has some serious enterprise functionality.
Subscribe to:
Posts (Atom)