Update (09/25/2009):

Vizioncore’s new version of vRanger (4.1.0 build 11581) has solved this issue. If you had experienced these issues you are advised to upgrade as soon as Vizioncore has released this new version officially!

Update (08/28/2009):
Vizioncore announced that the new version where I suppose this problem is fixed in will be released Mid-September.

Original Twitter-message:

Vizioncore vRanger Pro 4.1 DPP to be released in Mid-Sept. More info @ VMworld 2009 and VirtualVizion http://tinyurl.com/vc-vvizion

Since not too long I have been having problems with a crashing Virtual Center. The version where the problems started with was 2.5 Update 4. The problem was caused by vRanger Pro 4.0 DPP. Please keep on reading to see the complete story on this problem.

Problem occurrence and symptoms:

In short this is how the problems first occurred: our DBA installed SP3 on our SQL 2005 Server and rebooted our SQL Server afterwards which normally wouldn’t be much of a problem but in this case our Virtual Center wouldn’t start anymore. The service starts and crashes after 5 seconds. The vpxd.log shows an Win32_Exception and some debug data and that’s it.

Attempts to solve the problem:

At first the problems appeared to have been caused by the update on the SQL Database Server, so these are the steps I’ve taken in my attempts to solve the problem.

• The things we’ve tried to solve this mysterious problem: Reinstalled VC against our production database: same problem
• Reinstalled VC against our production database on another SQL Sever
  with SQL Server 2005 SP2 on it: same problem
• Reinstalled VC with a clean database and reconfigured our entire
  environment, but after a reboot of the VC: same problem except this
  time we installed VC 2.5 Update 5 instead of Update 4.
• Cleanly installed a new server with a fresh Windows 2003 install and
  all recent updates and installed VC 2.5 Update 5 again with a clean
  database but the same problem occurred after rebooting the VC server

We repeated the last step probably about 2 times and it was really making me desperate for a solution because we have a pretty big environment.

The actual problem:

After desperately creating a Support Request at VMware they asked me to disable vRanger if we had that running in our environment because it was known to cause problems to Virtual Center.

The thing that already raised questions on my behalf before this incident was that the new vRanger constantly kept an open connection to Virtual Center, and as soon as you killed that session from Virtual Center the vRanger service immediately reconnected. So it appeared to me that during the startup of the Virtual Center service it constantly tried to connect and initiate something or tried to retrieve data from it which caused the Virtual Center service to crash right after startup.

Resolution:

After disabling the vRanger service –which wasn’t installed on our Virtual Center server itself by the way- we held our breath and restarted our Virtual Center server, and surprisingly the service started flawlessly.
To make sure it really was vRanger we re-enabled the vRanger service and restarted the Virtual Center service and it crashed immediately.

After disabling the vRanger service VC functioned flawlessly again.

Final note:

The road to the solution has taken around two weeks. It was horrible. Finally the solution is known. By the way, previous versions of vRanger do NOT cause these problems!

Update (helpful reaction from Vizioncore, see comments below for original message):

It’s not hard to create a PowerShell script to list your current snapshots, but of course it’s a nice challenge to create a oneliner that’ll take care of this.

I’ve created several oneliners with different output, for instance: this one is to show snapshots of your total VMware infrastructure:

Get-VM | Sort Name | Get-Snapshot | Where { $_.Name.Length -gt 0 } | Select
VM,Name,Description,Created

While in fact in the organisation I work for we have several locations throughout the country and several VMware ESX Servers running on each one which we’ve divided in different Datacenters in our VMware Infrastructure.

So for instance you can also use the following line to list all Snapshots in a specific Datacenter:

Get-Datacenter -Name "Your Datacenter" | Get-VM | Sort Name | Get-Snapshot | 
Where { $_.Name.Length -gt 0 } | Select VM,Name,Description,Created

Or just for one specific cluster:

Get-Cluster "Cluster Name" | Get-VM | Sort Name | Get-Snapshot | Where
{ $_.Name.Length -gt 0 } | Select VM,Name,Description,Created

All of these oneliners give somewhat the same output, which looks like this (company sensitive names are sensored, sorry )

You can also add several other cmdlets to change the output of the result data, such as  ConvertTo-HTML or Out-GridView (if you use PowerShell 2.0 already).

When using NFS for your datastores on your SAN (eg. NetApp) and you resize your datastore on the fly it always takes a while before the ESX/vSphere servers refresh the datastore so you can see the new size of the volume.

This can be annoying when you want to create a VM on one of these volumes and ESX still thinks the volume is too small because it hasn’t refreshed it’s datastores yet. Normally I would pick one ESX server where I want to put the VM on for starters and refresh the NFS volume which I want to use:

Ofcourse I wanted to make it easier for myself so I wrote a little PowerShell script to refresh the datastores on all ESX Servers in our Cluster. I think it’s also possible to make a oneliner out of this and I will try to create one and if I succeed I’ll post it again

$vcHost = "virtual.center.host"
$vcClusterName = "Your ClusterName"
$vcUser = "username"
$vcPass = "password"

$vcServer = Connect-VIServer -Server:$vcHost -User:$vcUser -Password:$vcPass -WarningAction:SilentlyContinue

$vmHosts = Get-Datacenter | Get-Cluster -Name:$vcClusterName | Get-VMHost | Sort

ForEach ($vmHost in $vmHosts) {
    If (Get-VMHostStorage -VMHost:$vmHost -Refresh -ErrorAction:SilentlyContinue -WarningAction:SilentlyContinue) {
        Write-Host $vmHost.Name," host storage information refreshed"
    } else {
        Write-Host $vmHost.Name," host storage refresh failed!"
    }
}

Disconnect-VIServer -Server:$vcServer -Confirm:$false

After the script has been running the datastores on all your ESX-servers in the cluster will be refreshed so all resizes  of your NFS volumes on your SAN will be visible and “active”.