All about VMware, NetApp, PowerShell, Windows, Microsoft and Linux in general.

NOTE: I published this article a bit earlier, but due to the fact that this was a not-yet-released feature I removed this post once again. Since this feature is released for a long time already I deciced to re-publish it, enjoy!
–

Last night I got a message from our monitoring system that one of our NFS-volumes on our NetApp SAN was almost full but at this particular moment I was driving home and by the time I got home the NFS volume was completely full which resulted in the Virtual Machines freezing up.

After sizing up the NFS volume the machines do not automatically start to ‘unfreeze’ again, and they were waiting for me to answer the question in the vSphere client to “Retry” or “Abort”. Ofcourse I could’ve just answered “Retry” on each machine to get them going again, but I thought it would be nicer to solve this with the almighty PowerCLI since it were like 30 Virtual Machines.

Get-Cluster "Your Cluster" | Get-VM | Sort | Get-VMQuestion | `
Set-VMQuestion -Option "Retry" -Confirm:$false

This will result in answering all machines waiting to continue with the “Retry” option in your cluster (in this example I used “Your Cluster” as clustername).

I hope you won’t be needing it much, but if you do it will be handy

Just a small oneliner to display all the servers, their overall status, CPU and Memory usage in all your Datacenters (can be handy if you have multiple datacenters).

Get-Datacenter | Sort | Get-VMHost | Sort | Get-View |`
Select Name, OverallStatus, `
@{N="CPU Usage (GHz)";E={[math]::round(
$_.Summary.QuickStats.OverallCpuUsage/1024,2)}}, `
@{N="Memory Usage (GB)";E={[math]::round(
$_.Summary.QuickStats.OverallMemoryUsage/1024,2)}} 

And it will give you an output that looks like this:

You may not find it very useful like this, but you can also add a Where statement to this line to filter on several things. For example, you can decide you only want to see the servers that have yellow or red Overall Status due to high memory or CPU usage:

Get-Datacenter | Sort | Get-VMHost | Sort | Get-View | `
Select Name, OverallStatus, `
@{N="CPU Usage (GHz)";E={[math]::round(
$_.Summary.QuickStats.OverallCpuUsage/1024,2)}}, `
@{N="Memory Usage (GB)";E={[math]::round(
$_.Summary.QuickStats.OverallMemoryUsage/1024,2)}} | `
Where { $_.OverallStatus -ne "green" }

Which will give you something like this:

Ofcourse these onliners are cool and handy to use, but you can also use these oneliners to write a script around it to monitor your servers. I will post a script like that soon to show you different interpretations of this script.

When using NFS for your datastores on your SAN (eg. NetApp) and you resize your datastore on the fly it always takes a while before the ESX/vSphere servers refresh the datastore so you can see the new size of the volume.

This can be annoying when you want to create a VM on one of these volumes and ESX still thinks the volume is too small because it hasn’t refreshed it’s datastores yet. Normally I would pick one ESX server where I want to put the VM on for starters and refresh the NFS volume which I want to use:

Ofcourse I wanted to make it easier for myself so I wrote a little PowerShell script to refresh the datastores on all ESX Servers in our Cluster. I think it’s also possible to make a oneliner out of this and I will try to create one and if I succeed I’ll post it again

$vcHost = "virtual.center.host"
$vcClusterName = "Your ClusterName"
$vcUser = "username"
$vcPass = "password"

$vcServer = Connect-VIServer -Server:$vcHost -User:$vcUser -Password:$vcPass -WarningAction:SilentlyContinue

$vmHosts = Get-Datacenter | Get-Cluster -Name:$vcClusterName | Get-VMHost | Sort

ForEach ($vmHost in $vmHosts) {
    If (Get-VMHostStorage -VMHost:$vmHost -Refresh -ErrorAction:SilentlyContinue -WarningAction:SilentlyContinue) {
        Write-Host $vmHost.Name," host storage information refreshed"
    } else {
        Write-Host $vmHost.Name," host storage refresh failed!"
    }
}

Disconnect-VIServer -Server:$vcServer -Confirm:$false

After the script has been running the datastores on all your ESX-servers in the cluster will be refreshed so all resizes  of your NFS volumes on your SAN will be visible and “active”.

Some time ago someone we agreed on putting a minimum of 15 Virtual Machines on each NFS-volume on our NetApp storage. Now, almost a year later we have about 14 NFS volumes and the spreading of the VM’s on the Volumes have become a bit out of the ordinary. Mostly because we didn’t have an easy overview of how many VM’s there are present on each volume or because VM’s needed to be created on a short notice.

Since I’ve been busy with PowerShell I decided to create a nice script/utility to create this overview and to increase our insight in our NFS volumes. At first I made a console script, but soon -while I was discovering the posibilities of PowerShell- I created a GUI for it using PrimalForms.

List NFS Store Usage - Example Output

The source is no rocket science, but I just found the utility very handy/helpful to monitor our storage environment for VMware. If you intend to use this sourcecode, don’t forget to adjust the VirtualCenter hostname and access credentials. Also, the names of our DataStores are all start with “VMWare_NFSxx” and I have a condition running to check if the DataStore name starts with that. You might want to adjust that to your datastore name convention.

Read more