Using Independent Disks in vCloud

Yesterday I wrote about the PowerShell module I’ve written (CIDisk.psm1) to allow manipulation of independent disks in a vCloud environment. This post shows some usage options and details some of the caveats to be aware of when using disks in this manner.

My test environment has two VMs (named imaginatively ‘vm01’ and ‘vm02’), and the VDC they are in has access to four different storage profiles (‘Platinum’, ‘Gold’, ‘Silver’ and ‘Bronze’ storage). The default storage policy for the VDC is ‘Bronze’, but what if we want to create independent disks on other profiles? The -StorageProfileHref parameter to New-CIDisk lets us do this. Once connected to our cloud (Connect-CIServer) we can find the Hrefs of the available storage profiles we can use:

Let’s create 2 independent disks, a 10G disk on ‘Platinum’ storage and a 100G disk on ‘Silver’ storage:

We can see in the vCloud interface that these disks now exist in our VDC (Note: you may have to completely refresh your vCloud session using your browser’s refresh before the ‘Independent Disks’ tab appears):

There are no context actions for these disks though and we can’t attach/detach them to VMs in the vCloud interface.

Our VM01 virtual machine currently has a 40GB base disk attached and no other storage:

 

We can mount both our new independent disks to this VM using the following:

Looking at the VM01 Hardware tab following this shows both disks mounted:

Note again that no manipulation options are available in the vCloud UI, but at least it’s obvious that independent disks have been attached to VM01.

After rescanning storage in the guest, we can see the new storage devices on VM01:

And once these are brought online, initialized, storage volumes created and drive letters assigned, we can use the disks inside the guest (the volume names don’t get automatically mapped – I’ve just named the volumes the same as the independent disk objects for consistency):

At this point everything appears to be working fine, but there can be a catch here – if you restart the virtual machine you may find that the server attempts to boot from one of the newly mounted independent disks. Luckily vCloud Director 8.10 allows us to get into the VM BIOS and change the boot order settings:

Once restarted into BIOS we can select the correct boot order:

With the server restarted, we can create some test content in ‘disk01-plat’ to prove that the data moves when we reattach this disk to VM02:

And to dismount ‘disk01-plat’ from VM01 and mount it to VM02 we can:

Looking at the available storage in VM02 after a disk rescan shows our disk has transfered across:

Finally, checking the contents of the ‘E:\’ drive shows our test folder & file have made it across:

And Get-CIDisk can be used to verify the disk attachments after moving disk01 to VM02:

Hopefully this gives a better idea of how CIDisk can be used to manage independent disks in a vCloud environment, it would be nice if VMware included the management functions in the UI, but for now at least you can use PowerShell to easily achieve the same results without having to write against the API directly.

As always, any comments / feedback greatly appreciated.

Jon

Detailed VM Storage Information in vCloud Director

I recently had a request from one of our customers who wanted an easy / scriptable method to determine the storage allocations on their hosted VMs in our vCloud platform, preferably from PowerShell. That should be easy I thought and set about my usual Google-based research. I initially found this post from Alan Renouf which I forwarded back to the client.

Unfortunately, while this achieved part of the answer, this particular customer had a number of VMs which had hard disks attached using multiple/different storage profiles and they wanted to get the details of these too. So I set about writing some code to see if I could get full storage information about the VM and all of its disks. I ended up having to access the vCloud REST API directly for this information but it wasn’t too bad.

First, I created a ‘worst-case’ test VM where the 3 attached hard disks which were created one each on our ‘Gold’, ‘Silver’ and ‘Bronze’ storage policies:

test02-hardware-properties

(Just to make sure everything would work I also created the 3 disks on 3 different storage Bus Types). I also set the VM storage policy to something different:

test02-general-properties

My first step was a function to access the vCloud REST API, I found this post from Matt Vogt’s blog which had some code for this which I shamelessly borrowed (hey, why reinvent the wheel unless you need to):

The return from the Get-CIVM cmdlet includes a reference to the VM object within the vCloud API:

Using this we can obtain our disk information:

Filtering the returned RasdItemsList for a ResourceType of 17 (Hard Disk), we can get a list of attached hard disks:

So this gets us to a point where we have all of the hard disk information, but how do we find the storage policy for each disk? It turns out that each disk has an attribute ‘HostResource’ which provides the URI to the storage policy from which the disk has been allocated:

So how can we convert the storageProfileHref values into meaningful (human readable) storage profile names? We can use another API call to establish the name of each vdcStorageProfile:

Querying the API for every vdcStorageProfile for every disk is going to generate a lot of calls for any significant number of VMs, so in the code below I’ve added a hash stored in a global variable which caches these results so that any storageProfileHref which has been seen before doesn’t need to generate an additional API call.

Putting it all together

So we now have a way of determining all of the information we need, using PowerShell custom objects allows us to write a function which returns all of our VM and storage details in a easily consumable form for further processing.

The script included at the bottom of this article produces the following output for my test environment containing 2 VMs of which the ‘pxetest01’ VM has no disks attached:

It can also return just the disk information as another custom object:

And we can check the number of disks attached to any VM:

Finally because the output is a PowerShell object, we can easily turn this custom object into JSON for use in further processing:

Hopefully you’ve found this post useful, let me know in the comments if you have any issues or would like to see more examples like this.

Jon.

Full script to find storage policy information for vCloud VMs using the vCloud REST API:

 

Live import VMs to vCloud Director

Tom Fojta wrote a great blog post about the new capability in vCloud Director 8.10 to import running VMs into vCloud Director. This is a huge asset in migration scenarios where customers can’t afford outages when being migrated into the vCD environment. Unfortunately the API syntax to actually initiate the import is a little convoluted and not the easiest process to manage.

I set about writing a PowerShell script to significantly simplify the process of initiating a live-import operation. The script itself is available from github at the following link: https://github.com/jondwaite/vcdliveimport.

The liveimport.ps1 script contained in this repository does the following:

  • Prompts for a credential to be used to connect to both vCloud Director (System context) and vCenter – if you have different usernames/passwords for each you’ll need to adjust this.
  • Enumerates the available vCenter instances registered as Provider Virtual Datacenters (PVDCs) in vCloud Director and allows one to be selected as the source vCenter for the migration.
  • Lists the available VMs in the selected vCenter instance, filters this list based on selectable criteria (e.g. don’t offer to import ‘Guest Introspection’ VMs) and allows the source VM to be selected.
  • Lists available destination Virtual Datacenters (VDCs) in the vCloud Director environment and allows the destination VDC to be selected.
  • Displays the appropriate POST request information to be submitted to vCloud Director to initiate the live-import of this VM.
  • Optionally – Submits the REST API request directly to the vCloud Director environment to actually initiate the import process.

An example transcript of this process is show below. Hopefully this helps someone else out and helps to make it easier for you to live-import running VMs into vCloud Director.

Jon.

Example Session Transcript:

 

Create an empty vApp in vCloud Director

Sometimes you just need to create a new vApp with no contents at all – maybe for testing, or maybe you want to populate it with VMs built ‘from scratch’ rather than cloned from templates. This is easy to do in the vCloud Director web UI – you just skip the addition of any VM templates or new VMs and can easily create empty vApps, but how about programatically?

The VMware documentation is remarkably slim in this regard – all the documented methods I could find for vApp creation require either cloning from existing vApp templates, from existing VMs or from uploaded OVF files.

So how do we create a brand-new empty vApp? Turns out it’s pretty simple – once you discover the ‘composeVApp’ method on an Organization VDC supports creation of empty vApps.

If using the REST API we can simply create an XML body document of type ‘composeVAppParams’ and submit it against the OrgVDC’s /action/composeVapp link.

An example XML document body could be:

<?xml version=”1.0″ encoding=”UTF-8″?>
<ComposeVAppParams
name=”MyEmptyVapp”
xmlns=”http://www.vmware.com/vcloud/v1.5″
xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″>
<Description>My vApp Description</Description>
<AllEULAsAccepted>true</AllEULAsAccepted>
</ComposeVAppParams>

We then ‘POST’ this document body to the link: ‘https://<Cloud Server DNS name or IP address>>/api/vdc/<ID of our VDC>/action/composeVApp’ not forgetting to add a header of ‘Content-Type: application/vnd.vmware.vcloud.composeVAppParams+xml’ to the POST request.

If we want to accomplish the same thing using PowerShell / PowerCLI it’s easy too (once connected to our cloud using Connect-CIServer):

$vapp = New-Object VMware.VimAutomation.Cloud.Views.ComposeVAppParams
$vapp.Name = “MyEmptyVapp”
$vapp.Description = “My vApp Description”
$myorgvdc = Get-OrgVdc -Name ‘My OrgVDC Name’
$myorgvdc.ExtensionData.ComposeVApp($vapp)

No idea if this is ‘officially’ supported or not – so use at your own risk and be aware that the implementation could change in a future release and break this (although I’d be surprised as this is almost certainly the action that the vCD web UI is submitting ‘behind the scenes’ when you manually create an empty vApp).

Jon.

Client Integration Plugin madness

One of the frustrations dealing with the vSphere Web Client has always been the requirement for a browser plugin to import/export OVF templates. In vSphere 6 and vCloud Director 8 this has reached a whole new level of frustration. The issue is that both vSphere 6 and vCloud Director 8 offer to download and install a package called ‘VMware-ClientIntegrationPlugin-6.0.0.exe’. In an ideal world this plugin once correctly installed would work for the OVF import/export/upload/download functionality in both products… right?

Meanwhile in this world, although the package names are identical, the functionality is not – if you’ve installed the vSphere version then you can’t upload/download OVFs or ISOs in vCloud Director and if you’ve got the vCD variant installed then vSphere OVF import/export doesn’t work. Uninstalling and reinstalling the ‘correct’ version fixes the problem (until you need the ‘other’ one again), but can be easier said than done – particularly in the case of shared desktop server administration environments where other users having a browser session open will prevent reinstalling browser plugins.

So how do you tell the two packages apart? In the current releases of vSphere 6.0Update1 and vCloud Director v8.0 the packages are significantly different sizes:

vCloud Director 8 vSphere 6.0 Update 1
Package Filename VMware-ClientIntegrationPlugin-6.0.0.exe VMware-ClientIntegrationPlugin-6.0.0.exe
File Version 11.0.0.2826 10.0.0.3637
Product Version 6.0.0.2826 6.0.0.3637
File Size 48.8 MB 94.9 MB

So the easiest way to tell is the smaller 48.8 MB file is vCD and the larger 94.9 MB one is vSphere.

If (as I do) you often need to use both versions then maybe consider setting up separate management desktops (or virtual apps) for each so you can easily reach one that’s going to work for you.

Hopefully VMware will fix this in a future release and provide a single integration plugin that works across both products.

Update – 18th March 2016
VMware have just released vCloud Director for Service Providers v8.0.1 (http://pubs.vmware.com/Release_Notes/en/vcd/801/rel_notes_vcloud_director_801.html) which appears to have reverted the vCD Client Integration product to version 5.6.0 – there is also mention in the release notes on the possible clashes between vSphere and vCD client integration toolsets so it appears that VMware are at least aware of the issue.