Detailed VM Storage Information in vCloud Director
I recently had a request from one of our customers who wanted an easy / scriptable method to determine the storage allocations on their hosted VMs in our vCloud platform, preferably from PowerShell. That should be easy I thought and set about my usual Google-based research. I initially found this post from Alan Renouf which I forwarded back to the client.
Unfortunately, while this achieved part of the answer, this particular customer had a number of VMs which had hard disks attached using multiple/different storage profiles and they wanted to get the details of these too. So I set about writing some code to see if I could get full storage information about the VM and all of its disks. I ended up having to access the vCloud REST API directly for this information but it wasn’t too bad.
First, I created a ‘worst-case’ test VM where the 3 attached hard disks which were created one each on our ‘Gold’, ‘Silver’ and ‘Bronze’ storage policies:
(Just to make sure everything would work I also created the 3 disks on 3 different storage Bus Types). I also set the VM storage policy to something different:
My first step was a function to access the vCloud REST API, I found this post from Matt Vogt’s blog which had some code for this which I shamelessly borrowed (hey, why reinvent the wheel unless you need to):
1function Get-vCloudREST($href) {
2 $request = [System.Net.HttpWebRequest]::Create($href)
3 $request.Accept = "application/*+xml;version=20.0" # For vCloud Director 8.xx or later API
4 $request.Headers.add("x-vcloud-authorization",$global:session.sessionID)
5 $response = $request.GetResponse()
6 $streamReader = new-object System.IO.StreamReader($response.getResponseStream())
7 $xmldata = $streamReader.ReadToEnd()
8 $streamReader.Close()
9 $response.Close()
10 return $xmldata
11}
The return from the Get-CIVM cmdlet includes a reference to the VM object within the vCloud API:
1PowerCLI C:\> $test02 = Get-CIVM -Name 'test02'
2PowerCLI C:\> $test02 | fl
3ExtensionData : VMware.VimAutomation.Cloud.Views.Vm
4Status : PoweredOff
5Deleted : False
6GuestOsFullName : Other Linux (64-bit)
7CpuCount : 1
8MemoryMB : 384
9MemoryGB : 0.375
10VMVersion : v11
11Org : MyOrg
12OrgVdc : MyOrgVDC
13VApp : test02
14Description :
15Href : https://<cloud.com>/api/vApp/vm-ef4a4594-c631-421a-98f0-4d15670c98ad
16Id : urn:vcloud:vm:ef4a4594-c631-421a-98f0-4d15670c98ad
17Name : test02
18Client : /CIServer=<user>:<org>@<cloud.com>:443/
19Uid : /CIServer=<user>:<org>@<cloud.com>:443/CIVM=urn:vcloud:vm:ef4a4594-c631-421a-98f0-4d15670c98ad/
Using this we can obtain our disk information:
1PowerCLI C:\> $queryHref = $test02.Href + "/virtualHardwareSection/disks"
2PowerCLI C:\> $xml = Get-vCloudREST($queryHref)
3PowerCLI C:\> $xml
4xml RasdItemsList
5--- -------------
6version="1.0" encoding="UTF-8" RasdItemsList
Filtering the returned RasdItemsList for a ResourceType of 17 (Hard Disk), we can get a list of attached hard disks:
1PowerCLI C:\> $xml.RasdItemsList.Item | Where { $_.ResourceType -eq 17 }
2
3AddressOnParent : 0
4Description : Hard disk
5ElementName : Hard disk 1
6HostResource : HostResource
7InstanceID : 2000
8Parent : 2
9ResourceType : 17
10VirtualQuantity : 8589934592
11VirtualQuantityUnits : byte
12
13AddressOnParent : 0
14Description : Hard disk
15ElementName : Hard disk 2
16HostResource : HostResource
17InstanceID : 2016
18Parent : 3
19ResourceType : 17
20VirtualQuantity : 8589934592
21VirtualQuantityUnits : byte
22
23AddressOnParent : 0
24Description : Hard disk
25ElementName : Hard disk 3
26HostResource : HostResource
27InstanceID : 16060
28Parent : 4
29ResourceType : 17
30VirtualQuantity : 8589934592
31VirtualQuantityUnits : byte
So this gets us to a point where we have all of the hard disk information, but how do we find the storage policy for each disk? It turns out that each disk has an attribute ‘HostResource’ which provides the URI to the storage policy from which the disk has been allocated:
1PowerCLI C:\> $disks = $xml.RasdItemsList.Item | Where { $_.ResourceType -eq 17 }
2
3PowerCLI C:\> $disks.Count
43
5
6PowerCLI C:\> $disks[0].HostResource
7
8vcloud : http://www.vmware.com/vcloud/v1.5
9storageProfileHref : https://<my cloud URI>/api/vdcStorageProfile/f89df73a-2fa5-40e4-9332-fdd6e29d36ac
10busType : 6
11busSubType : lsilogic
12capacity : 8192
13iops : 0
14storageProfileOverrideVmDefault : true
15
16PowerCLI C:\> $disks[1].HostResource
17
18vcloud : http://www.vmware.com/vcloud/v1.5
19storageProfileHref : https:// <my cloud URI>/api/vdcStorageProfile/d14c6c2e-2cfb-4ffe-9f31-599c3de42150
20busType : 6
21busSubType : lsilogicsas
22capacity : 8192
23iops : 0
24storageProfileOverrideVmDefault : true
25
26PowerCLI C:\> $disks[2].HostResource
27
28vcloud : http://www.vmware.com/vcloud/v1.5
29storageProfileHref : https:// <my cloud URI>/api/vdcStorageProfile/dc382284-bc65-4e61-b85f-ea7facba63e4
30busType : 20
31busSubType : vmware.sata.ahci
32capacity : 8192
33iops : 0
34storageProfileOverrideVmDefault : true
So how can we convert the storageProfileHref values into meaningful (human readable) storage profile names? We can use another API call to establish the name of each vdcStorageProfile:
1PowerCLI C:\> $sp = Get-vCloudREST($disks[0].HostResource.storageProfileHref)
2PowerCLI C:\> $sp.VdcStorageProfile.name
3Gold Storage Profile
4
5PowerCLI C:\> $sp = Get-vCloudREST($disks[1].HostResource.storageProfileHref)
6PowerCLI C:\> $sp.VdcStorageProfile.name
7Silver Storage Profile
8
9PowerCLI C:\> $sp = Get-vCloudREST($disks[2].HostResource.storageProfileHref)
10PowerCLI C:\> $sp.VdcStorageProfile.name
11Bronze Storage Profile
Querying the API for every vdcStorageProfile for every disk is going to generate a lot of calls for any significant number of VMs, so in the code below I’ve added a hash stored in a global variable which caches these results so that any storageProfileHref which has been seen before doesn’t need to generate an additional API call.
Putting it all together
So we now have a way of determining all of the information we need, using PowerShell custom objects allows us to write a function which returns all of our VM and storage details in a easily consumable form for further processing.
The script included at the bottom of this article produces the following output for my test environment containing 2 VMs of which the ‘pxetest01’ VM has no disks attached:
1PowerCLI C:\> $vmobjs
2
3VMName : pxetest01
4Id : urn:vcloud:vm:45a86f24-1109-4ead-881d-f865c1f0692f
5Status : PoweredOff
6vRAM : 1
7vCPU : 1
8HWVersion : v11
9vAppName : PXE Demo
10GuestOS : CentOS 4/5/6/7 (64-bit)
11Disks :
12
13VMName : test02
14Id : urn:vcloud:vm:ef4a4594-c631-421a-98f0-4d15670c98ad
15Status : PoweredOff
16vRAM : 0.375
17vCPU : 1
18HWVersion : v11
19vAppName : test02
20GuestOS : Other Linux (64-bit)
21Disks : {@{Name=Hard disk 1; StorageProfile=Gold Storage Profile; Quantity=8589934592; QuantityUnits=byte}, @{Name=Hard disk 2; StorageProfile=Silver Storage Profile; Quantity=8589934592; QuantityUnits=byte}, @{Name=Hard disk 3; StorageProfile=BronzeStorage Profile; Quantity=8589934592; QuantityUnits=byte}}
It can also return just the disk information as another custom object:
1PowerCLI C:\> $test02vm = $vmobjs | Where {$_.VMName -eq 'test02'}
2PowerCLI C:\> $test02vm.Disks
3
4Name StorageProfile Quantity QuantityUnits
5---- -------------- -------- -------------
6Hard disk 1 Gold Storage Profile 8589934592 byte
7Hard disk 2 Silver Storage Profile 8589934592 byte
8Hard disk 3 Bronze Storage Profile 8589934592 byte
And we can check the number of disks attached to any VM:
1PowerCLI C:\> ($test02vm.Disks | Measure).Count
23
Finally because the output is a PowerShell object, we can easily turn this custom object into JSON for use in further processing:
1PowerCLI C:\> $test02vm | ConvertTo-Json
2
3{
4 "VMName": "test02",
5 "Id": "urn:vcloud:vm:ef4a4594-c631-421a-98f0-4d15670c98ad",
6 "Status": "PoweredOff",
7 "vRAM": 0.375,
8 "vCPU": 1,
9 "HWVersion": "v11",
10 "vAppName": "test02",
11 "GuestOS": "Other Linux (64-bit)",
12 "Disks": {
13 "value": [
14 {
15 "Name": "Hard disk 1",
16 "StorageProfile": "Gold Storage Profile",
17 "Quantity": 8589934592,
18 "QuantityUnits": "byte"
19 },
20 {
21 "Name": "Hard disk 2",
22 "StorageProfile": "Silver Storage Profile",
23 "Quantity": 8589934592,
24 "QuantityUnits": "byte"
25 },
26 {
27 "Name": "Hard disk 3",
28 "StorageProfile": "Bronze Storage Profile",
29 "Quantity": 8589934592,
30 "QuantityUnits": "byte"
31 }
32 ],
33 "Count": 3
34 }
35}
Hopefully you’ve found this post useful, let me know in the comments if you have any issues or would like to see more examples like this.
Jon.
Full script to find storage policy information for vCloud VMs using the vCloud REST API:
1$global:spolrefs = @{}
2# Change to be appropriate for your scenario:
3$cloudURL = "<cloud FQDN or IP Address>"
4$cloudOrg = "<cloud Organization name>"
5
6# Check if we are already logged-in (interactive session) and if not, prompt for login:
7if (!$session.IsConnected) {
8 $creds = Get-Credential -Message "Authenticate to $cloudOrg Cloud Service"
9 $session = Connect-CIServer -Server $cloudURL -Org $cloudOrg -Credential $creds
10}
11
12# PS Function to query the vCloud Director REST API
13# ('Borrowed' from Matt Vogt's blog at: http://blog.mattvogt.net/)
14function Get-vCloudREST($href)
15{
16 $request = [System.Net.HttpWebRequest]::Create($href)
17 $request.Accept = "application/*+xml;version=20.0" # For vCloud Director 8.xx or later API
18 $request.Headers.add("x-vcloud-authorization",$global:session.sessionID)
19 $response = $request.GetResponse()
20 $streamReader = new-object System.IO.StreamReader($response.getResponseStream())
21 $xmldata = $streamReader.ReadToEnd()
22 $streamReader.Close()
23 $response.Close()
24 return $xmldata
25}
26
27# Function to get Storage Profile name from its Href (if not already known/cached in a global hash)
28# if already known we don't need to call the API again and can just return the value from the hash
29function Get-SPName($storage_href)
30{
31 if ($global:spolrefs.ContainsKey($storage_href)) {
32 return $global:spolrefs.Get_Item($storage_href)
33 } else {
34 $sp = Get-vCloudREST($storage_href)
35 $global:spolrefs.Add($storage_href, $sp.VdcStorageProfile.name)
36 return $sp.VdcStorageProfile.name
37 }
38}
39
40# Function to retrieve the disk information for a VM:
41function Get-VMDisks($VM)
42{
43 $VMDisks = @() # Start with an empty array
44 $queryHref = $VM.Href + "/virtualHardwareSection/disks" # API path for VM disk information
45 $xml = Get-vCloudRESt($queryHref)
46 $disks = $xml.RasdItemsList.Item | Where-Object {$_.ResourceType -eq 17} # Resource Type 17 = Hard disk drive
47 foreach ($disk in $disks)
48 {
49 $sp = Get-SPName($disk.HostResource.storageProfileHref)
50 $diskobj = New-Object -TypeName PSObject
51 $diskobj | Add-Member -Type NoteProperty -Name "Name" -Value ([String]$disk.ElementName)
52 $diskobj | Add-Member -Type NoteProperty -Name "StorageProfile" -Value ([String]$sp)
53 $diskobj | Add-Member -Type NoteProperty -Name "Quantity" -Value ([Int64]$disk.VirtualQuantity)
54 $diskobj | Add-Member -Type NoteProperty -Name "QuantityUnits" -Value ([String]$disk.VirtualQuantityUnits)
55 $VMDisks += $diskobj
56 }
57 return $VMDisks
58}
59
60function Get-VMInfo($VM)
61{
62 $vmobj = New-Object -TypeName PSObject
63 $vmobj | Add-Member -Type NoteProperty -Name "VMName" -Value ([String]$VM.Name)
64 $vmobj | Add-Member -Type NoteProperty -Name "Id" -Value ([String]$VM.ExtensionData.Id)
65 $vmobj | Add-Member -Type NoteProperty -Name "Status" -Value ([String]$VM.Status)
66 $vmobj | Add-Member -Type NoteProperty -Name "vRAM" -Value ([Decimal]$VM.MemoryGB)
67 $vmobj | Add-Member -Type NoteProperty -Name "vCPU" -Value ([Int]$VM.CpuCount)
68 $vmobj | Add-Member -Type NoteProperty -Name "HWVersion" -Value ([String]$VM.VMVersion)
69 $vmobj | Add-Member -Type NoteProperty -Name "vAppName" -Value ([String]$VM.VApp.Name)
70 $vmobj | Add-Member -Type NoteProperty -Name "GuestOS" -Value ([String]$VM.GuestOsFullName)
71 [PSCustomObject]$disks = Get-VMDisks($VM)
72 $vmobj | Add-Member -Type NoteProperty -Name "Disks" -Value $disks
73 return $vmobj
74}
75
76# Get all VMs in our Cloud Organization:
77$vms = Get-CIVM
78
79# Initialise an array for returned objects:
80$vmobjs = @()
81
82# For each VM, build our object:
83foreach ($vm in $vms) {
84 $vmobjs += Get-VMinfo($vm)
85}
86
87# Output our object to console:
88$vmobjs