Vizioncore vRanger 4.0 Pro Crashes vCenter 2.5 Update 3, 4 and 5
Update (09/25/2009):
Vizioncore’s new version of vRanger (4.1.0 build 11581) has solved this issue. If you had experienced these issues you are advised to upgrade as soon as Vizioncore has released this new version officially!
Update (08/28/2009):
Vizioncore announced that the new version where I suppose this problem is fixed in will be released Mid-September.Original Twitter-message:
Vizioncore vRanger Pro 4.1 DPP to be released in Mid-Sept. More info @ VMworld 2009 and VirtualVizion http://tinyurl.com/vc-vvizion
Since not too long I have been having problems with a crashing Virtual Center. The version where the problems started with was 2.5 Update 4. The problem was caused by vRanger Pro 4.0 DPP. Please keep on reading to see the complete story on this problem.
Problem occurrence and symptoms:
In short this is how the problems first occurred: our DBA installed SP3 on our SQL 2005 Server and rebooted our SQL Server afterwards which normally wouldn’t be much of a problem but in this case our Virtual Center wouldn’t start anymore. The service starts and crashes after 5 seconds. The vpxd.log shows an Win32_Exception and some debug data and that’s it.
Attempts to solve the problem:
At first the problems appeared to have been caused by the update on the SQL Database Server, so these are the steps I’ve taken in my attempts to solve the problem.
- The things we’ve tried to solve this mysterious problem: Reinstalled VC against our production database: same problem
- Reinstalled VC against our production database on another SQL Sever
with SQL Server 2005 SP2 on it: same problem - Reinstalled VC with a clean database and reconfigured our entire
environment, but after a reboot of the VC: same problem except this
time we installed VC 2.5 Update 5 instead of Update 4. - Cleanly installed a new server with a fresh Windows 2003 install and
all recent updates and installed VC 2.5 Update 5 again with a clean
database but the same problem occurred after rebooting the VC server
We repeated the last step probably about 2 times and it was really making me desperate for a solution because we have a pretty big environment.
The actual problem:
After desperately creating a Support Request at VMware they asked me to disable vRanger if we had that running in our environment because it was known to cause problems to Virtual Center.
The thing that already raised questions on my behalf before this incident was that the new vRanger constantly kept an open connection to Virtual Center, and as soon as you killed that session from Virtual Center the vRanger service immediately reconnected. So it appeared to me that during the startup of the Virtual Center service it constantly tried to connect and initiate something or tried to retrieve data from it which caused the Virtual Center service to crash right after startup.
Resolution:
After disabling the vRanger service –which wasn’t installed on our Virtual Center server itself by the way- we held our breath and restarted our Virtual Center server, and surprisingly the service started flawlessly.
To make sure it really was vRanger we re-enabled the vRanger service and restarted the Virtual Center service and it crashed immediately.
After disabling the vRanger service VC functioned flawlessly again.
Final note:
The road to the solution has taken around two weeks. It was horrible. Finally the solution is known. By the way, previous versions of vRanger do NOT cause these problems!
Update (helpful reaction from Vizioncore, see comments below for original message):
We are working very diligently to get this fix in the hands of our users, we have a fix that already out of our development process and is now in the hands of our QA team. You can also reference the following Knowledgebase article here http://www.vizioncore.com/support/knowledgebase/index.php and search for KB 00000296 about this issue. This KB will also be updated shortly for a root cause once the issue is fully cleared by development and QA, for those that are interested in the technical details about the issue. Thanks again for everyone’s support! And we hope to get everyone taken care of very soon!
Seeing the same exact thing in our environment. This has been super painful as we were desperately waiting v4 to fix so many of the problems with the previous version.
Disclaimer – I’m a vendor. If you’re interested in trying Veeam Backup & Replication as an alternative, we’re offering a special competitive replacement discount right now. We’d love the opportunity to show you our product. Thanks.
How are you backing up if you have vRanger disabled??!?1
@Rob Bergin
We downgraded to the previous version of vRanger.
@VMCarrie
No thanks. Sooner or later the fix for vRanger will apppear, I’m sure. So, until then we will use the previous version of vRanger which does the trick until then.
Mate I have suffered from this problem today although I’m running VC2.5U3 and found the problem to be with vRangerPro4 exactly as you did find. What a terrible day pheeeew.
I have posted a warning in VMware Communities forum (http://communities.vmware.com/thread/227074?tstart=0) and on my blog (http://deinoscloud.wordpress.com/2009/08/19/virtualcenter-service-to-core-dump-with-vrangerpro4/)
@VMCarrie, no software vendor are 100% bug free, not even Veeam. Now let see how fast Vizioncore will fix this bug…
Cheers,
@PiroNet
I also placed a warning forum and a reference to this blogpost on the VMware Communities forum earlier today:
http://communities.vmware.com/message/1341593
We are working very diligently to get this fix in the hands of our users, we have a fix that already out of our development process and is now in the hands of our QA team. You can also reference the following Knowledgebase article here http://www.vizioncore.com/support/knowledgebase/index.php and search for KB 00000296 about this issue. This KB will also be updated shortly for a root cause once the issue is fully cleared by development and QA, for those that are interested in the technical details about the issue. Thanks again for everyone’s support! And we hope to get everyone taken care of very soon!
@PiroNet
Sure, no vendor is 100% bug free, but at least I expect every enterprise solution to be 100% safe for the infrastructure. I did not expect to see such a severe bug that ruins virtual infrastructure from the company which positions itself as virtualization market leader. I thought vRanger is about disaster recovery, but apparently it is about making disasters…
Otra solución es instalar veeam Backup
@Josep
Please speak English so that everyone understands what you’re saying.
I guess you’re saying that the solution is to install Veeam Backup. That would be solution indeed, but I’d like to call that a ‘workaround’ and not really a solution to this problem.
The real and most likely solution -in my opinion- would be to disable vRanger 4 and/or downgrade to the previous version to have some continuity in your backup process and wait for the fix to be released from Vizioncore to fix vRanger 4.
If anyone is interested and you would like to read about the technical details, you can revisit the above KB, Thanks!
@Jason Mattox
Is the following solution mentioned in the “What is VMware or Vizioncore doing to resolve this issue?” section of the KB-article already implemented in the 4.0 DPP version? Or will it be fixed in the upcoming 4.1 release in Mid-Sept.?
Quote from KB article:
To combat this Vizioncore has added in additional API checks into vRanger Pro DPP to ensure that Virtual Center is fully ready for its VMware provided API’s. If not, vRanger Pro will wait, disconnect and reconnect to Virtual Center and try again until Virtual Center is fully started and is not providing a false positive about its readiness to serve VMware provided API’s, this will remove the issue with vRanger 4.0 and Virtual Center 2.5x.
It would be in the upcoming 4.1 release in Mid-Sept.
Glad to hear that Vizioncore will release a ‘patch’ or would I say an product update for mid-September. There is a simple workaround available but guys the bug is hanging around for +8 weeks now I would expect a patch released before a product update. Anyway better than a crappy patch quickly released and creating more bugs than it fixes…
@Josep, downgrade is not an option here. The differences between both products are huge. Just to mention that VCB is not used anymore in DPP.
I hit the same problem yesterday!
The following logs describe the Problem!
[2009-09-10 19:46:58.388 'App' 2156 error] Win32 exception: Access Violation (0xc0000005)
[2009-09-10 19:46:58.388 'App' 2156 error] Write (1) at address 0×000094
[2009-09-10 19:46:58.388 'App' 2156 error] eip: 0x17669fb esp: 0x3e3f680 ebp: 0x3e3f680
[2009-09-10 19:46:58.388 'App' 2156 error] eax: 0x3e3f694 ebx: 00000000 ecx: 00000000 edx: 0x3e3f601 edi: 0x781c5a4 esi: 0x3e3f730
[2009-09-10 19:46:58.481 'App' 2156 error]
vRange becomes more and more worse. We had so much trouble in the past with Version 4. I think we will change to anther vendor soon!
@PiroNet
I definately agree that the patch or release should really come out soon. Vizioncore is taking really long to fix this -in my opinion- rather serious issue.
It _WILL BE_ Mid-September next week, so I’m really anxious if they will make it in time!
Well, it might help to look how these replication production compare. Check out how vizionCore esxReplicator compare to Veeam backup & replication at:
http://www.itcomparison.com/DR/VizioncorevsVeeam/VizioncorevsVeeam.htm
I hope that help some one,
Enjoy,
Erick