locked
VMs fail to start - 'Storage Virtualization Service Provider' driver ?

    Question

  • Hi,

    After a re-boot the VMs fail to start with the following errors in the Hyper-V log. Apparently there have been no human config changes to rollback.

    I have un-installed and reinstalled Hyper V but no luck.

     

    'Storage Virtualization Service Provider' driver required by the Virtual Machine Management service is not installed or is disabled. Check your settings or try reinstalling the Hyper-V role.”

     

    'CNBVS008': Failed to open attachment 'V:\Hyper-v\CNBVS008\CNBVS008.vhd'. Error: 'A device attached to the system is not functioning.' (0x8007001F). (Virtual machine ID 94603FE7-4214-4BBA-B042-527B69F776B4)

    Microsoft Emulated IDE Controller (Instance ID {83F8638B-8DCA-4152-9EDA-2CA8B33039B4}): Failed to Power on with Error 'A device attached to the system is not functioning.' (0x8007001F). (Virtual machine ID 94603FE7-4214-4BBA-B042-527B69F776B4)

    Any ideas or trouble shooting

    Thanks
    Ray

    Tuesday, September 29, 2009 7:24 AM

Answers

  • Hi Sean,

    I have rebuilt the server as 2008 R2 and it appears OK.... Though shortly after the rebuild I have moved the VMs off and onto a 2008 R2 HA Cluster on Certified Hardware. 

    The Dell 2950 does indeed have a tape drive/library connected via a Dell SAS 5e controller (not on supported hardware list).

    I think I was a bit rash in my decision to put 2008 R2 onto our backup server, I didn't appreciate the difference between 2008 and R2, and lack of initial vendor support. Symantec Backup Exec for Windows Servers is also not fully supported under 2008 R2. I will rebuild with Server 2008 until certified by Symantec and Dell.

     

     

    Thanks for the response.

    Ray

    Thursday, October 08, 2009 9:20 PM

All replies

  • Are there any ISOs attached to the VMs?
    Is the VM attached to the physical DVD of the Host?

    This might be an issue with devices on the IDE bus (the bottom error), not necessarily the VHD.

    Brian Ehlert (hopefully you have found this useful)
    Tuesday, September 29, 2009 3:00 PM
  • I've got a ticket in with Microsoft support for the same exact issue. Did you shut down the VM prior to restarting the host? Do you have antivirus installed on the VM? I suspect if you attempt to inspect the vhd disk it will complain about that too. Thus far I have encountered this condition three times on the same server. The only method I was successful with was to completely reinstall the host OS, create a new VM and attach the old vhd file. It appears you have that located on a different drive than the OS so you could give that a shot if you don't mind the work involved. I will update this post if I hear any joy from Microsoft on this issue. Will be speaking to them later today. Good luck!
    Tuesday, September 29, 2009 4:11 PM
  • Hi,

    All VMs are down. I have removed all attached ISO from

    The VMs weren't shutdown prior to reboot. They came up in a saved state as per their setting.  The VMs are on SAN and local SAS HDD. I am able (and have) copied the VHDs to other servers. Note - I can't export the VM, same error.

    It does have Symantec End Point protection (AV only) 11.2.   The only different configuration of this server to our other servers is this one has Symantec Backup Exec (Tape back backup) and extra SAS controller connected to tape lib.

    Server is Dell 2950 and EMC San (latest driver, software, and firmware). It was a in place upgrade from 2008 SP2 64 bit Datacentre to 2008 R2 Datacentre. 

    I'll try rebuild.

    Ray

     

      
    Tuesday, September 29, 2009 11:38 PM
  • Hi Ray

    My name is Sean Dwyer and I'm a Support Escalation Engineer for the Windows Cluster and Virtualization team.
    I'm actively working a couple of customer cases that may be related to the same problem you're experiencing.

    I've got a couple of questions that will help me verify this theory:
    - Was the rebuild successful in removing the problem condition? Or has it returned after a couple of reboots?
    - When attempting to manage any VHD file, or create any VHD file through Disk Management under R2, have you seen the following message:
      "A virtual disk support provider For the specified File was not found"
    - Are there any tape devices attached to the R2 machine?
    - Please list all of the attached storage controllers in the 2950.
    - Do you have files in the \windows\system32\drivers directory called LSI_SAS.SYS and TPFILTER.SYS?

    - If you have a tape drive attached, can you reproduce the problem managing VHD files when the tape device is powered OFF during a machine cold boot? Meaning, power off the device, power off the computer, wait a few seconds, then power on the computer, but leave the tape device powered OFF for the duration of the test.


    As a side-note, based on my previous work with other customers, I want to mention that there are only 5 Dell controllers certified for R2. The list is here: http://www.windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext=&bCatID=1362&cpID=543&avc=10&ava=31&avq=0&OR=1&PGS=25&ready=0
    I'm not trying to 'pile on' with the issue you're already battling, but I want you to be aware of this supportbility issue so you aren't surprised at a later date. Dell has already informed my customers that they are unsupported with those controllers when using R2 as they've not yet finished WHQL testing the devices and drivers.

    I look forward to your reply!

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, October 08, 2009 1:53 PM
  • Hi Sean,

    I have rebuilt the server as 2008 R2 and it appears OK.... Though shortly after the rebuild I have moved the VMs off and onto a 2008 R2 HA Cluster on Certified Hardware. 

    The Dell 2950 does indeed have a tape drive/library connected via a Dell SAS 5e controller (not on supported hardware list).

    I think I was a bit rash in my decision to put 2008 R2 onto our backup server, I didn't appreciate the difference between 2008 and R2, and lack of initial vendor support. Symantec Backup Exec for Windows Servers is also not fully supported under 2008 R2. I will rebuild with Server 2008 until certified by Symantec and Dell.

     

     

    Thanks for the response.

    Ray

    Thursday, October 08, 2009 9:20 PM
  • Hi Sean,

    I have rebuilt the server as 2008 R2 and it appears OK.... Though shortly after the rebuild I have moved the VMs off and onto a 2008 R2 HA Cluster on Certified Hardware. 

    The Dell 2950 does indeed have a tape drive/library connected via a Dell SAS 5e controller (not on supported hardware list).

    I think I was a bit rash in my decision to put 2008 R2 onto our backup server, I didn't appreciate the difference between 2008 and R2, and lack of initial vendor support. Symantec Backup Exec for Windows Servers is also not fully supported under 2008 R2. I will rebuild with Server 2008 until certified by Symantec and Dell.

     

     

    Thanks for the response.

    Ray


    Thank you for your reply Ray.

    Regarding R2, there is some initial lag with storage vendor support as there's a general need for them to re-test their drivers with the new storage model that R2 runs with that incorporates VHD manipulation. It shouldn't be long until you'll see many more controller options available for R2.

    I'm happy to hear the VMs are happily running in a highly available environment! I hope you enjoy the 2008 Clustering experience. It's quite a difference over 2003!

    Here are a couple of links that may provide some additional information for you regarding HAVM's:
    Ask the Core (Setup/Cluster/Virtualization)Team Blog!
    http://blogs.technet.com/askcore/

    Hyper-V Security Guide
    http://technet.microsoft.com/en-us/library/dd569113.aspx?ca=SEC&su=SEC&sa=HYPV&ct=WEBS&cn=MSATN&au=ITPRO&go=HYPVTN&dt=04152009

    Clustering and High Availability
    --------------------------------
    http://blogs.msdn.com/clustering/archive/2009/05/07/9593050.aspx

    If the problem returns for you, be sure to let us know so we can look into it with you. Have a great weekend!


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, October 09, 2009 4:17 PM
  • I am having similar issues. I am using some syetsm in a lab not server clase, but have used them for VMware OK. I have a clean R2 install and have Hyper-V role set up and when I go to creeta  a VM and it when it goes to build the VHD I get "The system failed to create 'name of vhd here'. Error Code: A virtual disk support provider for the specified file was not found." I ran 2008 R1 Hyper-V fine on both these systems, I rebuilt it from scrath becasue after the R2 upgrade Hyper-V woudl have the same issue, I thought it was related to upgrade. I did clean R2 install and loaded Hyper-V. This system does have an attached tape drive. But it had the same tape drive on R1 Hyper-V as well. Seems Hyper-V is still way too new for my blood.
    Sunday, November 29, 2009 6:20 PM
  • I just tried powering off the tape drive and rebooting and the error is gone. Turn tape drive back on and reboot and it fails again. I am using an HP LTO autoloader using the MS drivers in the OS, why does the Ms drivers create this problem? I have opedn a premier case with MS support as I now have to choose, go back to R1 (not really an option) or stay R2 and decide between using Hyper-V and not backing up the system, or not using Hyper-V and being able to back up the system!
    Sunday, November 29, 2009 7:02 PM
  • Any luck with this? I have the same issue on 2008 R2 Hyper-V(PowerEdge 2950) with all of my guests. I am using an Adaptec 39160 for my tape device and the built in Perc 5/i for local storage. The Adaptec card is not supported but the drivers from 2008 non R2 worked. Everything was fine until I rebooted and got the Failed to open attachment  error on all of my guests. Based on the above suggestions I disabled the Adaptec card(s) in device manager and then rebooted. The guests started. I enabled the Adaptec card(s) in device manager and restarted the backup exec services. BE is now also working. I think the conflict is with the MS VHD HBA device under storage controllers but I don't know what to do about it

    Friday, December 04, 2009 5:24 AM
  • Update

     

    We’ve become aware of an issue with FSDepends.sys and certain backup software packages. 

    This issue is currently under investigation.


    In situations such as these being reported here, FSDepends.sys is probably set to Manual in the Registry. To workaround the problems, this driver
    should be set to Boot by following the directions below.

     

    Make sure that you back up the registry before you modify it. Make sure that you know how to restore the registry if a problem occurs. For more information about how to back up, restore, and modify the registry, click the following article number to view the article in the Microsoft Knowledge Base:

    322756 How to back up and restore the registry in Windows

     

    To workaround the issue, do the following:

    1)    Start Regedit.exe

    2)    Navigate to the following reg key: “HKLM\SYSTEM\CurrentControlSet\Services\FsDepends”

    3)    Under the “FsDepends” key change REG_DWORD value “Start” from “3” to “0”

    4)    Reboot the machine

    Let us know if this resolves your issue. Thank you.


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, December 17, 2009 5:06 PM
  • Sean,

    We experienced this issue about 9 hours after your post on 17-Dec.  I'm happy to report that the reconfiguration of the fsdepends.sys driver startup mode has resolved this issue in our case. 

    The host server boots, all backupexec 12.5 services installed on the host server start, and all VMs start properly.  Inspecting the drives of the VMs also works as it should.
    Monday, December 21, 2009 5:35 PM
  • That's excellent news, JD. Thank you.
    I'll be sure to post here when the investigation has reached its end.
    Thank you very much for following up to let us know that you're no longer encountering this issue!
    This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, December 21, 2009 9:37 PM
  • Confirmed the registry change to the FSDepends.sys is the easiest fix to this problem I have found to date.

    The commonality on two of my systems is also an Adapter SCSI card, Tape Library and Backup Exec.

    I previously removed Backup Exec 12.5 and installed Backup Exec 2010 Beta. This turned out to be only a temporary fix.
    Gary Herbstman Byte Solutions, Inc. http://bytesolutions.com
    Saturday, January 09, 2010 12:15 PM
  • I've had to change the FsDepends value as well on 'one' of my hosts.
    I have 2 identical hosts running W2K8 R2 , with 1 difference:
         - The Host where I did not had to change the FsDepends value run Backup Exec 12.5 SP3 (and hotfixes) connected to th External LTO-4 (HP1760) via a HP P212/ZM SAS card.
         - The Host where i had to change the FsDepends value, runs the same hardware and Software...except this time the server connects via a HP AH627 U320e SCSI to a HP MSL2024 LTO-4 library.

    Further to GaryH his post, I also only see this on the config with a 'tape library' not on the standalone tape drive (although one is a SAS and the other a SCSI U320 card)

    In the process of planning Backup Exec 2010 deployment  to solve other issues  on the VM's as well

    Thursday, February 11, 2010 1:17 PM
  • Thank you for the information Jurgen.

    We've put out an article that addresses the symptoms and workaround for this issue while it's still being investigated.

    Error message when you manage a VHD file in Windows Server 2008: "A Virtual Disk Provider for the specified file was not found”
    http://support.microsoft.com/kb/2013544


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, February 11, 2010 1:26 PM
  • Please note that we have now hit this issue on a Intel S5000PAL based system after upgrading the BIOS and firmware to the latest version. After rebooting the server it has this error and no VMs will migrate to it. What is important is that this only occured after upgrading the firmware, previously this server has run for months with no issues at all. The server is dedicated to the Hyper-v role, no tape drives are attached and nor is backup exec or similar backup software installed. Therefore, this issue clearly affects more than just tape drive users.

    The registry fix given in the support article does not resolve the issue in our case.

    Wednesday, March 10, 2010 5:35 PM
  • Morning Mrphil45

    What version bios did you start from, and which did you upgrade to?
    What version firmware did you start from, and which did you upgrade to?
    What is the exact message when you try to create a VHD in Disk Management?
    What is the exact message when you attempt to migrate a VM to this upgraded machine?

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, March 11, 2010 12:02 PM
  • I also have seen this issue within our Dell 2950 hardware environment.  The Registry key did not resolve the siutation.
    What did resolve the situation was:

    1.  Uninstall the Windows Backup Features and reboot
    2.  Mount the vhd under Disk Management.  Choose Update Drivers in Device Manager for the disk, as it was not automagically detected.  <very important>
    3.  Then run through the add VM Wizard in Hyper-V.

    Tuesday, March 16, 2010 9:33 PM
  • Please note that we have now hit this issue on a Intel S5000PAL based system after upgrading the BIOS and firmware to the latest version. After rebooting the server it has this error and no VMs will migrate to it. What is important is that this only occured after upgrading the firmware, previously this server has run for months with no issues at all. The server is dedicated to the Hyper-v role, no tape drives are attached and nor is backup exec or similar backup software installed. Therefore, this issue clearly affects more than just tape drive users.

    The registry fix given in the support article does not resolve the issue in our case.


    Hi. I have same issue with Intel S5000PALSATAR MB. Basic hardware , Intel MB with 2xsamsung 500gb sata RAID1 and no any extras stuff.

     

    hardware and 2008r2 are new installation.. no any other roles yet , only hyper-v.

    installed 2008r2 and intel server management and raid management.

    MB bios is nevest version

    No help in registry fix.

     

    BIOS:latest 98.

    s5000.86b.12.00.0098 date 6/23/2009


    I saw somewhere that Intel management software uninstall helped,, didn't helped

    Wednesday, March 31, 2010 9:32 PM
  • I've also encountered this problem, with an Intel S5000VSASATAR motherboard. In the hope that it might help isolate the cause, here are a few details: I performed a clean install of 2008 R2 on the system, with the OS on a pair of mirrored drives, using the onboard Intel SATA RAID controller. I configured my VMs on a separate single SATA drive, attached to the same controller. Everything worked fine.

    At that point, I added an Intel RAID activation key (AXXRAKSW5) to enable RAID5. I then added a RAID5 array with three disks, and copied my VMs from the single drive to the new array. I also upgraded to the latest version of the BIOS for the S5000 motherboard. When I tried to launch HyperV, I saw the same message noted above: "virtual disk support provider not found."

    Moving the VMs back to the single drive didn't help, and I also discovered that I couldn't create new virtual disks, just as others have described previously.

    Thursday, April 08, 2010 11:40 PM
  • I'm having almost the same issue on my deployment server.

    Hardware is S5000VSASATA, 2008 R2, 8Gb FB Kingston, ESRTII MIRROR (onboard RAID1 for OS) and LSI 8704 ELP RAID10 (raid card for data), no tape drives, and two X5260 Xeons

    • I have tried changing FSdepends and VHDMP startup to "0" (Auto)
    • I flashed the BIOS/BMC back to previous release (BIOS/BMC from 98/47 to 96/46)
    • Reloading the role several times

    I belive the issues started when CPUs have been downgraded, but could be wrong since the F/W was flashed in process. I get a series of four errors in hyper-V every time I try to create or attach a VHD to new or existing VM:

    Microsoft-Windows-Hyper-V-VMMS/Admin

    'WS1' failed to add device 'Microsoft Virtual Hard Disk'. (Virtual machine ID C67A2162-3397-42FB-98DD-9D237122189E) Event: 14140

    'WS1': Cannot open attachment 'D:\Hyper-V\WS1\VS1.vhd'. Error: 'A virtual disk support provider for the specified file was not found.' (7864368). (Virtual machine ID C67A2162-3397-42FB-98DD-9D237122189E) Event:12140

     

     

    'WS1': Cannot open attachment 'D:\Hyper-V\WS1\VS1.vhd'. Error: 'A virtual disk support provider for the specified file was not found.' (7864368). (Virtual machine ID C67A2162-3397-42FB-98DD-9D237122189E) Event: 12140

     

    'WS1': Cannot get information for attachment 'D:\Hyper-V\WS1\VS1.vhd'. (Virtual machine ID C67A2162-3397-42FB-98DD-9D237122189E) Event: 12620

    Wednesday, April 14, 2010 10:16 PM
  • Same issue here also.

    Intel S5000VSA running BIOS S5000.86B.06.00.0074.020220071613 (02/02/2007)

    Worked fine under 2008 R1, migrated the machines away to an R2 box and reinstalled this one with R2, now cannot migrate machines back to it from either R1 or R2, or create new hard disks and mount them in Hyper-V.

    Can mount the VHD just fine in Server Manager > Disk Management > Attach VHD

    Memory - 12Gb DDR2 Kingston PC2-5300

    Storage - Adaptec RAID card, 3405 running latest BIOS and server running latest driver (updated today).

    Network - Dual onboard Intel Pro/1000 EB running Intel driver version 9.13.16.0 (dated 01/09/2009 but was downloaded in the ProSet in the last two weeks)

    Tried the registry fix, no change.

    No tape drives or backup software installed.

    Errors: "failed to add device 'Microsoft Virtual Hard Disk'. (Virtual machine ID EF5F7A4F-C1C8-497E-95F0-409276E673A4)"

    "Cannot open attachment 'e:\Machines\test.vhd'. Error: 'A virtual disk support provider for the specified file was not found.' (7864368). (Virtual machine ID EF5F7A4F-C1C8-497E-95F0-409276E673A4)"

    "Cannot get information for attachment 'e:\Machines\test.vhd'. (Virtual machine ID EF5F7A4F-C1C8-497E-95F0-409276E673A4)"

    Permissions were changed on e:\machines to allow everyone read/write access as a test.

    Seems the S5000VSA is the common factor in recent posts to this thread.

     

    Tuesday, May 04, 2010 4:22 PM
  • You state: "Permissions were changed on e:\machines to allow everyone read/write access as a test."

    I just want to mention that "everyone" does not include the "System" or "Computer System" context, it only applies to users.  In this case the system needs the permissions.

    Have you tried creating a new VM using the existing VHD? (this is the best way to re-set the permissions)

    How did you "migrate" your VMs back to this host?  (there might be some type of key point there)

     


    Brian Ehlert (hopefully you have found this useful)
    Tuesday, May 04, 2010 5:39 PM
  • SYSTEM and Administrators already had full read/write. Now everyone has full read/write not just SYSTEM or Administrators.

    Creating a new VM with an existing VHD gives the same result as creating a new machine with a new VHD. If you use the Hyper-V new machine wizard to create a new machine, when it tries to create the disk it fails, then it deletes the machine (as the process did not complete). You can create a machine without a disk just fine. The VHD location is within the directory structure created by Hyper-V for the machine so all files should inherit the same permissions.

    Migrating was attempted using Microsoft System Center Virtual Machine Manager which we are currently evaluating. This system was fine under R1. We migrated all the servers off to an R2 box, then reinstalled this box with R2. Setup and Hyper-V role addition completed without incident, and SCVMM does not report any problems or necessary updates for this host. Under R1 there were no problems with running virtual machines or creating/adding drives. Other R1 to R2 migrations and R2 to R2 migrations were completed successfully using this tool, just not with this particular reinstalled server once it was on R2.

    Following the failed migration, creation of a new machine was attempted manually using the existing VHD file (which is no great problem if you know the RAM/CPU etc settings, you just have to reset the TCP/IP address back to what it was after the machine is booted and be prepared to remove orphaned network card IP settings from the registry afterwards).

    The common thread here seems to be the S5000 board from Intel as opposed to permissions on the files (which are currently all users, all contexts, SYSTEM, administrators, Everyone minimum read/write with full control permissions unchanged from the O/S defaults when the drive was formatted). Unfortunately the existing Microsoft guidance on this issue does not relate to the S5000 scenarios people have posted here as there are no tape drives installed. We tried the fix anyway, without any success. I have checked the certifications, the RAID card is certified under R2, the S5000 appears certified under R1. I do not know if the board is being evaluated for R2 or not, there seems no guidance either way. As it is an Intel provided server motherboard I am sure there are going to be a lot of people encountering this problem as they upgrade to R2 to try and realise the improvements.

    I am happy for someone from Microsoft to remote in and take a look should they wish to contact me.

    Wednesday, May 05, 2010 2:37 PM
  • Any further comments/suggestions?

    Do Microsoft have access to one of these boards to test with, or would you like to remote in?

     

    Friday, May 07, 2010 10:43 AM
  • The other problem is that this thread is marked as solved which is true for the original problem, but not the later problem. Should we start a new thread on this?
    Friday, May 07, 2010 10:44 AM
  • Hi, is there any updates about this issue? no new thread or so?
    Tuesday, June 08, 2010 9:29 PM
  • Hi, is there any updates about this issue? no new thread or so?

    Hi. Have you checked this http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/thread/b9c91808-67ee-4123-9d42-1f323d4d97e1.
    Wednesday, June 09, 2010 12:48 PM
  • Yes. Follow the link that Dimitar77 lists below.


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, June 09, 2010 1:00 PM