Automatically upgrade vCPU's

I am lazy. Well, not lazy, I just don't like working at ungodly hours. Here's the story: I got this request to upgrade 2 customer VM's from 3 vCPU's to 4 vcpu's. No biggy, I thought. But he wanted it done after 1 AM.. Wait, that's not during office hours!

OK, I could fix that with my trusty Powershell toolkit, and a scheduled task. But then I talked to the customer who went like: "Sure that's fine with me to script it, but make sure the sessions are drained from the webserver".. Hmm, F5 loadbalancers. I've not worked too much with those before, really. I get the general ideas within loadbalancing, but let's try and script against it.

So I had a number of things I wanted to happen:

• Log in to the F5 loadbalancer, preferably not with a script filled with credentials hardcoded into it. 
• Wait an X amount of time for connections to drop from the specific pool
• Shut down the machine, and wait for it to be properly turned off.
• Set the number of CPU's and memory. This had an extra challenge, namely that the person who built the VM put in 1 vCPU with 3 cores, instead of 3 vCPU's.
• Send an email with the results, so I can see what happened from my phone.

I'll spend a few posts on those different things.

Firstly the vCPU upgrade, which is the basis of this entire story:

Normally, you can turn off a VM, upgrade vCPU's, then start a VM again, through this simple set of commands:

$VM=Get-VM -Name 'WEB01'

Stop-VMGuest –VM $VM –Confirm:$False

do {

        $status = (get-VM $VM).PowerState

    }until($status -eq "PoweredOff")

$VM | Set-VM -NumpCPU 4 –Confirm:$False | Start-VM

However, since someone used 1 vCPU with multiple cores instead of multiple vCPU sockets, things work a little differently. If you would try this command, the VM would be shut down, but no upgrade would happen, and start back up with the same amount as before.

Fortunately there's another trick for that:

$VMSpec = New-Object -Type VMware.Vim.VirtualMachineConfigSpec -Property @{"NumCoresPerSocket" = 4;"numCPUs" = 4}

$VM.ExtensionData.ReconfigVM_Task($VMSpec)

(That first line shouldn't be cut off like that, should be 1 line, but alas: blog template gets in the way)

Now the VM gets upgraded to 4 vCPU's, although you would think it'd be 4x4 cpu's looking at the syntax.

So now the final complete code would be:

Add-PSsnapin VMware.VimAutomation.Core

Connect-VIServer myVcenter -ErrorAction Stop

$VM=Get-VM -Name 'WEB01'

Stop-VMGuest –VM $VM –Confirm:$False

do {

        $status = (get-VM $VM).PowerState

    }until($status -eq "PoweredOff")

$VMSpec = New-Object -Type VMware.Vim.VirtualMachineConfigSpec -Property @{"NumCoresPerSocket" = 4;"numCPUs" = 4}

Start-VM $VM

Yeay, success! Now on to the next bit...

Getting a list of vm's with OS'es

I wanted to do a Get-VM|select name, GuestOSFullname, but that doesn't work. So some googling gave me this little gem on the vmware forum:

New-VIProperty -Name GuestFullName -ObjectType VirtualMachine -ValueFromExtensionProperty 'Guest.GuestFullName' -Force

Now I can do a Get-VM | Select Name, GuestFullName and I get an overview of which machines have which OS.

I'm sure there are other ways of getting this too, but this was what I was looking for.

P.S., if I do Get-VM | Select Name,Guest , I do get the OS, but it is prepended with the hostname.

HP Firmware versions with PowerCLI

For documentation purposes  I needed BIOS and ILO versions of the vSphere environment.

I got the script from http://vnugglets.com/2011/11/get-hp-firmware-info-for-hp-vmware.html and combined some updates in the comments in the script so that it also shows which NetXen firmware and NetXen drivers are on the system. I adapted the script to give the info for all machines in the cluster, not just a single cluster.

Foreach ($strHostsClusterName in Get-Cluster){

Get-View -ViewType HostSystem -Property Name, Runtime.HealthSystemRuntime.SystemHealthInfo.NumericSensorInfo -SearchRoot (Get-View -ViewType ClusterComputeResource -Property Name -Filter @{"Name" = "^$([RegEx]::escape($strHostsClusterName))$"}).MoRef | %{

    $arrNumericSensorInfo = @($_.Runtime.HealthSystemRuntime.SystemHealthInfo.NumericSensorInfo)

    # HostNumericSensorInfo for BIOS, iLO, array controller

    $nsiBIOS = $arrNumericSensorInfo | ? {$_.Name -like "*System BIOS*"}

    $nsiArrayCtrlr = $arrNumericSensorInfo | ? {$_.Name -like "HP Smart Array Controller*"}

    $nsiILO = $arrNumericSensorInfo | ? {$_.Name -like "Hewlett-Packard BMC Firmware*"}

    $nsiNXdev = $arrNumericSensorInfo | ? {$_.Name -like "nx_nic device*"}

    $nsiNXdrv = $arrNumericSensorInfo | ? {$_.Name -like "nx_nic driver*"}

    # assume all at same level and take first or set as n/a

    if ( $nsiNXdev.Count -gt 0 ) {

       $nsiNXdevice = $nsiNXdev[0].Name

    } else {

       $nsiNXdevice = "n/a"

    }

    if ( $nsiNXdrv.Count -gt 0 ) {

       $nsiNXdriver = $nsiNXdrv[0].Name

    } else {

    $nsiNXdriver = "n/a"

    }

New-Object PSObject -Property @{

VMHost = $_.Name

"SystemBIOS" = $nsiBIOS.name

"HPSmartArray" = $nsiArrayCtrlr.Name

"iLOFirmware" = $nsiILO.Name

"nx_nic device" = $nsiNXdevice

"nx_nic driver" = $nsiNXdriver

    } ## end new-object

} ## end Foreach-Object

}

If I feel like it, I'll someday add LSI Logic cards and such as well. For now, it got me the info I needed, without logging on to 20 servers ;-)

Run the script from PowerCLI, by first connecting to the vCenter (Connect-Viserver <vcenter>) and then ./fwscript.ps1 | Export-CSV d:\fwlist.csv

CDP info from vSphere platform

For documentation purposes, I wanted to get an overview of all host connections to which switches they are connected, and which ports exactly. As luck would have it, VMware already has made some scripts to do this.

I adapted it to suit my needs:

$Hosts = Get-VMHost | sort -property Name

foreach ($vhost in $Hosts){

$vmh = Get-VMHost -Name $vhost

If ($vmh.State -ne "Connected") {

  Write-Output "Host $($vmh) state is not connected, skipping."

  }

Else {

  Get-View $vmh.ID | `

  % { $esxname = $_.Name; Get-View $_.ConfigManager.NetworkSystem} | `

  % { foreach ($physnic in $_.NetworkInfo.Pnic) {

    $pnicInfo = $_.QueryNetworkHint($physnic.Device)

    foreach( $hint in $pnicInfo ){

      # Write-Host $esxname $physnic.Device

      if ( $hint.ConnectedSwitchPort ) {

        $hint.ConnectedSwitchPort | select @{n="VMHost";e={$esxname}},@{n="VMNic";e={$physnic.Device}},DevId,PortId

        }

      }

    }

  }

}

}

To run this, simply copy this script to a file, open up powershell, connect to vcenter, and run it with:

.\VMHostCDPInfo.ps1 | Format-Table -AutoSize |Out-File cdpinfo.txt

It ignores all nics where it cannot retrieve any CDP info from, but other than that, you get a nice list, which you can put with your documentation which looks like:

VMHost          VMNic   DevId    PortId
------          -----   -----    ------
server01        vmnic0  switch01 GigabitEthernet1/25
server01        vmnic10 switch02 GigabitEthernet2/0/31

How to make ESXi crash

While looking at a VMware KBTV video, I saw a cool trick to make your ESXi host crash (which can be good for testing purposes):

From SSH, type: vsish -e set /reliability/crashMe/Panic 1

Apparently, I'm telling nothing new: Seaching for that string on Google, I found the links below for more info:

http://www.seancrookston.com/2012/01/09/forcing-a-kernel-dump-on-a-vsphere-host-the-purple-screen-of-death/

http://www.ntpro.nl/blog/archives/1388-Lets-create-some-Kernel-Panic-using-vsish.html

Unable to remove ESXi host from vCenter

I needed to remove an ESXi host from vCenter that I wanted to re-use, but I was unable to. After disconnecting the server, the "Remove" button was greyed out:

 Some searching gave the tip to remove it via PowerCLI (Remove-VMHost), but that did not work either:

It turned out the ESXi host could not be removed because it was part of a cluster.

Solution:

Move the ESXi host entry (which is disconnected) to the top of your vCenter tree, out of any cluster. Then re-run the Remove-VMHost command (most likely the Remove function will be visible in the GUI as well, but I just pressed up and enter in my PowerCLI screen, and it started to remove)

HP Servers disconnecting

We come across an issue lately, with several types of HP servers that have QLogic/NetXen NC375i networkcards in them. They disconnect, causing a disruption of service. You can imagine that having an NFS mount or iSCSI target with that happening is less than desirable and has caused Windows clusters to fail over and ESX/ESXi hosts to go crazy. This problem is solved by rebooting the host. This issue is very much OS independent!

In windows eventlog you may see things like:
DEVICE: HP NC375i Integrated Quad Port Multifunction Gigabit Server Adapter #4
PROBLEM: Tx path is hung. The device is being reset.

In ESX you see things in /var/log/vmkernel like:
Jul 31 21:02:12 server01 vmkernel: 165:01:40:09.914 cpu19:4295)<5>nx_nic[vmnic8]: Device is DOWN. Fail count[8]
Jul 31 21:02:12 server01 vmkernel: 165:01:40:09.915 cpu19:4295)<3>nx_nic[vmnic8]: Firmware hang detected. Severity code=0 Peg number=2 Error code=1 Return address=0


HP has brought out an advisory saying that indeed there are problems:

Network Adapters and Affected Firmware Versions

Network Adapter	Affected Firmware Versions
CN1000Q Dual Port Converged Network Adapter	EARLIER than firmware version 4.8.22
NC375i Integrated Quad Port Multifunction Gigabit Server Adapter	EARLIER than firmware version 4.0.585
NC375T PCI Express Quad Port Gigabit Server Adapter	EARLIER than firmware version 4.0.585
NC522m Dual Port Flex -10 10GbE Multifunction BL-c Adapter	EARLIER than firmware version 4.0.585
NC522SFP Dual Port 10GbE Server Adapter	EARLIER than firmware version 4.0.585
NC523SFP 10Gb 2-port Server Adapter	EARLIER than firmware version 4.9.81

The NC375i adapter is integrated on the following servers and storage systems:

• ProLiant DL370 G6 Server
• ProLiant DL580 G7 Server
• ProLiant DL585 G7 Server
• ProLiant DL980 G7 Server
• HP Business Data Warehouse Appliance
• StorageWorks D2D4312 Backup System
• StorageWorks D2D4324 Backup System

Servers manufactured after 1 april 2012 are not affected by this, but check the firmware level if you suffer from this issue. An older interface may still have this issue in your newer machine.

How to check the firmware version:

Windows:
Go to the HP network utilities, and click on the network interface you are having issues with, and click Properties. The Information tab will show the Boot Code, which is the firmware version:

Alternatively, you can run the update tool, and it will tell you which version you are currently running as well.

Linux:

Type "modinfo netxen_nic" and look for the firmware line.

[user@server-01 ~]$ modinfo netxen_nic | grep firmware

firmware: phanfw-4.0.579.bin   <--------  version 4.0.579, so needs an update

ESX/ESXi:
VMware have released a KB article to get the firmware and driver version, available here.

Resolution:

The resolution is to update the firmware of the network cards. The advisory lists the latest drivers and firmware. For Windows and Linux, there are proper update tools, but unfortunately for VMware, no firmware update utility is given, and the Linux firmware utility does not work.

On ESX/ESXi you have to make use of a Linux LiveCD and boot from it (ESX-server in Maintenance mode and reboot). In our case we used Novell SLES11 CD (free ISO download, registering necessary) as the Rescue-CD for RHEL5 gave several errors running the firmware update-utility. Perhaps a OpenSUSE, Fedora, Ubuntu or other distro LiveCD can be used as well, but we haven't tested those.

Many thanks go to my colleague Sven for the info :-)

Getting an overview of patches for your ESX hosts

A customer asked for an overview of which patches exactly were needed for his ESX hosts. He wanted to review the patches before they were applied with Update Manager. Unfortunately, you can't get the list from the gui in a nice way, so some PowerCLI goodness was needed.

First off, you need to have the Update Manager cmdlets installed to be able to use the Get-Compliance cmdlet in the code.

The following script will go over each host, and output the severity, patch ID, release date, additional info (a link to the kb article) and a short update to what the patch is for.

It will look something like this:

HostSecurity,ESX410-201204402-SG,"4/26/2012 10:00:00 AM","For more information, see http://kb.vmware.com/kb/2014988.","Updates libxml2"

Now for the code itself:

ForEach ($HostToCheck in Get-VMHost){

$Details = Get-Compliance $HostToCheck -Detailed| Select -ExpandProperty NotCompliantPatches| Select @{N="Hostname";E={$HostToCheck}}, Severity, IdByVendor, ReleaseDate, Description, Name

$ComplianceResult += $Details

}

$ComplianceResult | Export-CSV -Path c:\NeededPatches.CSV -NoType

Speeding up PowerCLI

As you may have seen, I like PowerCLI. Only thing is, it's not the fastest to start up, and running the first cmdlet takes some time. After that, it goes fine.

The first tip from the VMware PowerCLI Blog is to go to:

Windows > Control Panel > Internet Options > Advanced > Security > Check for publisher's certificate revocation and uncheck that box (also helps the SQL Server Management Studio start up faster if it is not connected to the internet).

Unfortunately, this did not help me so much with running the first cmdlet. So I was mighty glad that I came across this article from vNugglets to speed up PowerCLI 5. This led to another VMware PowerCLI Blog article with all the steps for earlier versions.

Basically, the issue is that due to the fact that the .Net framework compiles the underlying code on first use, so you can precompile things to speed up first run:

Open up an administrative command prompt, and run the following commands:

C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install "VimService41.XmlSerializers, Version=4.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install "VimService40.XmlSerializers, Version=4.0.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install "VimService25.XmlSerializers, Version=2.5.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

If you are running on 64-bit OS, you need to run the following as well:

C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ngen.exe install "VimService41.XmlSerializers, Version=4.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ngen.exe install "VimService40.XmlSerializers, Version=4.0.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ngen.exe install "VimService25.XmlSerializers, Version=2.5.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

This needs to be done once, and works for all users and applications.

I did a test on one setup, using the test  on the PowerCLI blog which is:

Measure-Command { Get-VMHost } | fl TotalSeconds

This test went from 24.4 seconds to 4.6 seconds! More than 5 times as fast!! Again, thanks go to vNugglets!

How much space are the VM's using?

Quick script to give me an overview of which VM has how much diskspace in use, and also how much is free.

Get-VM -name MYVM-* | Where { $_.PowerState -eq "PoweredOn" } |

Get-VMGuest |

Select VmName -ExpandProperty Disks | Select VmName, Path, Capacity, @{ N="PercFree"; E={ [math]::Round( ( 100 * ( $_.FreeSpace / $_.Capacity ) ),0 ) } }

| Out-File c:\temp\myvmspace.txt

Note that this is of only the VM's that start with "MYVM-". Change this according to whatever you want ( or remove the -name to just get all vm's, sizes and free space).

List the portgroups for the VM's

As part of a makeshift DR solution, one of my requirements is to have a list of vm's that shows which networkcard is in which portgroup. Powershell to the rescue!

Get-VM | Get-NetworkAdapter | Select-Object @{N="VM";E={$_.Parent.Name}},@{N="NIC";E={$_.Name}},@{N="Network";E={$_.NetworkName}}| Export-Csv vms_in_networks.csv

Hey presto, you have a list that shows which nic goes where.

Cloned VM's and WSUS

 When you clone Windows machines, you may run into the issue where WSUS updates are not working. Even though sysprep has been run, the wsus authorization appears to not have been reset with it. I use this script below to reset the Windows Update to a fresh state.

net stop wuauserv
del c:\windows\WindowsUpdate.log
REG DELETE "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v AccountDomainSid /f
REG DELETE "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v PingID /f
REG DELETE "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v SusClientId /f
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v LastWaitTimeout /f
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v DetectionStartTime /f
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v NextDetectionTime /f
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v AUState /f
net start wuauserv
wuauclt /resetauthorization /detectnow
pause

Paste this into a batchfile, run it, and wait a short while, and then the updates should start again. Note that this deletes the WindowsUpdate.log file. If you want to do troubleshooting of previous WSUS errors, either delete that line, or copy the file to a safe place. I delete it so I get a fresh view of what is happening with the Windows Updates.

You shouldn't virtualize *everything*

VMWare says you can virtualize everything, but that doesn't mean you should. One of those things you actually shouldn't (dare I say can't) virtualize, is NTP. If someone asks you to virtualize the NTP server, here are two nice links to give you why that's a bad idea:

http://support.ntp.org/bin/view/Support/KnownOsIssues

9.2.2. Xen, VMware, and Other Virtual Machine Implementations

NTP was not designed to run inside of a virtual machine. It requires a high resolution system clock, with response times to clock interrupts that are serviced with a high level of accuracy. No known virtual machine is capable of meeting these requirements.

Run NTP on the base OS of the machine, and then have your various guest OSes take advantage of the good clock that is created on the system. Even that may not be enough, as there may be additional tools or kernel options that you need to enable so that virtual machine clients can adequately synchronize their virtual clocks to the physical system clock.

http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

Page  18:

Using NTP in Linux and Other Guests

The Network Time Protocol is usable in a virtual machine with proper configuration of the NTP daemon.

The following points are important:

Do not configure the virtual machine to synchronize to its own (virtual) hardware clock, not even as a fallback with a high stratum number. Some sample ntpd.conf files contain a section specifying the local clock as a potential time server, often marked with the comment “undisciplined local clock.” Delete any such server specification from your ntpd.conf file

Continued: Storage vMotion via PowerCLI

I spoke before about Storage vMotion via PowerCLI, and today I got to put it into practice. It works, *but* there's a small thing: It doesn't do thin provisioning on the fly, as far as I can tell. So if you had a thick VM, and you wanted to make it thin, normally via the interface you get an option to do thin provisioning during the Storage vMotion. The Move-VM statement has no such feature yet (I think, at least I couldn't find it). Some searching around told me that the awesome LucD posted a function on poshcode to get the thin provisioning into Move-VM (sort of). I haven't tested it out yet, but it seems to be what I needed.


Oh and for my own reference: If I want to move all vm's from "cluster1" to the New-Storage datastore:

Get-Cluster cluster1 | Get-VM | Move-VM -Datastore(Get-Datastore "New-Storage")

Removing old hardware in Windows VM's

If you P2V (physical to virtual) a system, or even when you create a VM on your VMWare workstation and move that to an ESX host (V2V, virtual to virtual), old hardware is "left behind" in Windows. The goal of a VM is to make it use as little resources as possible, so it is best you clean that up. Secondly, if the hardware in question is an old network interface, the IP address on that interface still exists, so Windows starts moaning there's a duplicate IP address.

There's a relatively simple way of fixing that: First, open up a command prompt and type

set devmgr_show_nonpresent_devices=1

Next, type: devmgmt.msc

The device manager will start up. In there, go to the menu "View" and click on "Show hidden devices". Now all sorts of previously hidden hardware will show up. Right mouseclick on the old hardware you want to remove (the icons that are of a lighter color than the existing hardware).

Now in Windows 2008/2008R2, if you do the same with a command prompt, you will get nothing. Fix that issue by running the command prompt as an administrator.

vCenter: Don't forget the SQL Server Agent!

So this is one I learned the hard way: If you use SQL server for your vCenter installation, don't forget to start the SQL Server agent automatically!

vCenter installs rollup jobs which it needs to process the statistics to make it need less granular so it takes up less space.

These are called rollup jobs, and are controlled by the statistics in the vCenter server settings. I hear you ask: So what happens if I don't start SQL Server agent automatically?

Well.. Nothing, at first. But after a week, the statistics don't get rolled up, so the first table in the SQL database that is used for the statistics (VPX_HIST_STAT1) keeps growing and growing, and the data won't be processed into the next table, called VPX_HIST_STAT2, nor the next, etc. There are 4 tables. The last 3 tables will have no data, so when you look at the day, week, month or year statistics, you'll have no statistics. Starting up the SQL Server Agent quickly will give SQL time to roll up the data, but wait too long and you have too much data to process. There is a way to delete the data, described in this article, but you will lose all statistics.

So save yourself some trouble: If you are installing SQL Server for vCenter, don't forget the SQL Server Agent

TCPDump of NFS traffic that is using jumbo frames

I had an issue a while ago where I needed to troubleshoot poor NFS performance, so VMWare support asked me to do a tcpdump of the traffic. I was using jumbo frames and the performance was poor after about 5 minutes of not using the NFS.

On the vswitch that has the NFS VMKernel (you ARE using a dedicated vswitch for NFS, right?) create a new service console.

Use an unused IP address from the range that you use on your storage LAN. If you are using a different vlan, don't forget to set that too. You will be using the vswif that is created with this previous action to do the tcpdump. Go to the properties of the vSwitch and set promiscuous mode to "Accept".

Now to do a tcpdump of your traffic.

tcpdump -i vswif2 -w ./dump_1 -s 256 (-s 256 is a buffer of 256MB I believe)

You will get a lovely SMALL file.. Not what you want. This is because vswif2 (the newly created vswif) is not using jumbo frames. To get vswif2 on jumbo frames type the following in the command prompt:

ip link set dev vswif2 mtu 9000
service network restart vswif2

Now do the tcpdump again, and watch your harddrive fill up with data. Stop tcpdump with CTRL-C (and don't wait too long, because it fills up quickly).

VMWare data traffic with jumbo frames

Set the jumbo frames on vSwitch1 (which I am using for NFS). Log on to the console of the ESX host, and type:

esxcfg-vswitch -m 9000 vSwitch1

# remove the old vmkernel for storage which is on MTU 1500

esxcfg-vmknic -d "VMkernel - Storage"

# add the vmkernel back for storage again with MTU  9000 (vmkernel has ip 192.168.1.2)

esxcfg-vmknic -a -i 192.168.1.2 -n 255.255.255.0 -m 9000 "VMkernel - Storage"

Now the vSwitch1 and NFS vmkernel are using MTU 9000, and if your physical switch and SAN are configured for jumbo frames too, you should have a speed increase.

Pushing Host Profiles via PowerCLI.

Nice.. I had to push a host profile to a whole number of hosts, but they were all running a lot of VM's. Anyone who's seen vSphere knows that via the interface, you need to set a server in maintenance mode, apply the host profile, then take it out of maintenance mode. This is fine for one or two machines, but by the time you get to the 5th or 6th server, you get a bit bored, especially when you know you've got 20 to do.. So in comes Powershell (I should say PowerCLI) again, to save me some time.. I thought.

First things first: Figure out how to get a host profile:

$MyHostProfile = Get-VMHostProfile -Name "coolprofile"

Fail. Fail? Yes, fail. Somehow I run into a wall, with PowerCLI telling me that my profile can't be found. Some googling tells me that there's a bug in PowerCLI, but there's a workaround:

Get-VMHostProfile -Entity *

Woohoo! It shows me a hostprofile that was already pushed to the client before. So my statement would now become:

$MyHostProfile = Get-VMHostProfile -Entity * 

Note that this workaround would only work with one host profile. I'll figure out how to get a specific hostprofile implemented some day, but I've got one, so I got lucky (this time).

Now to apply a host profile to an esxhost:

Apply-VMHostProfile -Entity $esxhost -Profile $MyHostProfile -Confirm:$false

Damn, that was easy. Especially since the first part was so difficult..OK, now now to get a host in maintenance mode: 

Set-VMHost -VMHost $esxhost -State maintenance 

Great, that works! Now how do I get it out? 

Set-VMHost -VMHost $esxhost -State connected 


Cool, works too! Now I have to put the whole thing together:

$hosts = "esx1","esx2","esx5","esx6"
$MyHostProfile = Get-VMHostProfile -Entity *
foreach ($esxhost in $hosts) {
Set-VMHost -VMHost $esxhost -State maintenance
Apply-VMHostProfile -Entity $esxhost -Profile $MyHostProfile -Confirm:$false
Set-VMHost -VMHost $esxhost -State connected
}


As you can see, I used an array of hosts in this case (the first line of code. Didn't want all hosts in my case) but I could have changed that to all hosts by just doing a get-vmhost.

Running this now puts each host in maintenance mode, applies the host profile, and takes it out again.. Now imagine that for 20, 30 or even 100 hosts...... Yes, I *like* PowerCLI...

Storage VMotion via PowerCLI

I keep running into occasions where I need to storage vmotion a lot of VM's to different locations. I looked at a solution some time ago, but that was an old version of PowerCLI. Turns out nowadays it's really easy:

get-vm "MyVM"| move-vm -datastore(Get-Datastore "New-Storage")

This simply moves the MyVM virtual machine to the New-Storage datastore.. Remove the "MyVM", and suddenly ALL vm's move to the New-Storage datastore: SWEET. Gotta love PowerCLI, and especially since I've got a job coming up where I need to be moving a whole bunch of vm's to new storage....