none
Hyper-V Guest VM's disappeared after orderly reboot (Not Virus Scanning Issue)

    Question

  • I wanted to perform some long overdue maintenance on a hyper-v host.  My setup is two Win2008 R2 Core Hyper-V machines with local disk storage (no san, no virus protection on the hosts).  I powered down the Guest OS's,  exported them from  Virtual1 and copied the to Virtual2.   The guest OS came up and worked no problem on Virtual2.   Later on in the day I powered down Virtual2.  When virtual2 came back up 2 of the 3 guest VM's have disappeared from the management console.

    Some items of note...

    • There is no virus protection running on the hosts (others seem to have been having an issue with virus scanning).
    • Event logs show the orderly shutdown of the guest VM's,  event logs then show only 1 of the machines turning back on.
    • I see no useful errors in the event logs related to starting the machines back up
    • There is a single event log entry for "Hyper-v failed creating a new partition (status 11)".  Although, there is plenty of free RAM & Disk space

    Anyone have any idea what's going on?  I'm kind of baffled as where to even begin looking.

    Thanks for any help,

    -eric

     

     

     

    Tuesday, April 27, 2010 1:50 AM

Answers

  • Hi,

     

    If the symbolic here is still in "%systemdrive%\ ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\", the corresponding VM should appear in Hyper-V Manager console. Please remove the corresponding symbolic link from that folder(please backup them before remove) and then use the icacls command to create a new symbolic to check whether you get the same issue.

     

    By the way, if you are not using the build-in administrator, you need to run the above command in an elevated CMD prompt line.

     

     

    Best Regards,

    Vincent Hu

     

    Tuesday, May 04, 2010 8:07 AM
    Moderator

All replies

  • Hi,

     

    This error "Hyper-v failed creating a new partition (status 11)!" may not help. It is a general error and usually indicates that there is not enough resource (usually the memory) to start a virtual machine (creating a new partition). As you may know that, most of these hang problems relate to incompatible hardware or drivers, low resource or if too many applications are running and exhausting the resources.

     

    So I would like to confirm whether you have other applications running on the problematic Hyper-V computer?

     

    By the way, please navigate to "%systemdrive%\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines" to see if the symbolic link related to the GUID of the 3 VMs exists. You may need to uncheck the "Hide extensions for known file types" in the Advanced settings of the Folder Options, as these folders and files are hidden by default.

     

    If you haven't made any changes to the configurations files of the specific virtual machine, you can try the following steps to get it back. Otherwise, you may not get it back and you may need to create a new virtual machine with the existing VHD file.

     

    1. Open an elevated command prompt, then navigate to "%systemdrive%\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\" (it's a hidden path).

     

    2. Run the following command:

     

    Mklink GUID.xml D:\VM\Test-VM\Virtual Machines\GUID.xml

     

    3. Grant full access to the virtual machine’s security identity using the command:

     

    icacls "%SYSTEMDRIVE%\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\GUID.xml" /grant "NT VIRTUAL MACHINE\GUID":F /l

     

    Note: The GUID is the GUID of the specific virtual machine. You can get this number from the location where you put the configuration file (the .xml file). I am assuming that the path of this configuration file is "D:\VM\Test-VM\Virtual Machines\GUID.xml".

     

    In addition, please restart the Hyper-V host computer again to check whether you get the same issue(you may just start one VM this time).

     

    Best Regards,

    Vincent Hu

     

    Tuesday, April 27, 2010 7:39 AM
    Moderator
  •  

    > This error "Hyper-v failed creating a new partition (status 11)!" may not help

    Yep, well aware.  That's why I mentioned that I had enough Ram/disk.  Also the VM's were up and running, wouldn't a hardware incompatability prevent them from ever coming up on the new machine?

     

    > Confirm whether you have other applications running on the problematic Hyper-V computer

    Nope, it's a Win 2008 R2 CORE - only running hyper-v and basic file services

     

    > see if the symbolic link related to the GUID of the 3 VMs exists

    Yes, the symbolic links do exist in "%systemdrive%\ ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\"

     

    > please restart the Hyper-V host

    No affect, problem persists.


    > Grant full access to the virtual machine’s security identity

    Ok, interesting ....  icacls fails with the following error message :

    NT VIRTUAL MACHINE\C28E716A-C80B-4C10-9493-5AE28B4F0B8D: No mapping between account names and security IDs was done.
    Successfully processed 0 files; Failed processing 1 files

    • The VM was already running.  Is there a way the permissions could be reset because of an orderly reboot?  I have no idea how a reboot could change permissions.
    • I'm term serviced to the CORE machine--does the concept of an elevated command prompt even exist here?
    • This KB describes my failure with icacls, although it only seems to refer to win2003, I've seen mention that this issue applies to 2008. http://support.microsoft.com/kb/94787
    • EDIT -- just wanted to note that "C28E716A-C80B-4C10-9493-5AE28B4F0B8D" in the error message above is the service sid of one of the problematic virtual machines 

     

    I'm still kind of flummoxed here.  Now what?

    • Edited by Eric-Barr Tuesday, April 27, 2010 10:18 AM add a note about the icacls error
    Tuesday, April 27, 2010 10:10 AM
  • Hi,

     

    If the symbolic here is still in "%systemdrive%\ ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\", the corresponding VM should appear in Hyper-V Manager console. Please remove the corresponding symbolic link from that folder(please backup them before remove) and then use the icacls command to create a new symbolic to check whether you get the same issue.

     

    By the way, if you are not using the build-in administrator, you need to run the above command in an elevated CMD prompt line.

     

     

    Best Regards,

    Vincent Hu

     

    Tuesday, May 04, 2010 8:07 AM
    Moderator
  • Hello vincent. I have a similar problem with Icacls, I keep getting the same error message like my predecessor here.

    no matter if I run it as local admin or domain-admin. What to do?

     

    My problem is that I eagerly want to restore my VM, since export/import does NOT work (Export did not create any .exp-file...).

    I read some hints on how-to restore VMs here (http://eniackb.blogspot.com/2009/05/how-to-recover-hyper-v-virtual-machine.html) and was going with the third possibility there, until the error struck.

     

    any ideas?

     

    thanks in advance

    Martin Smarda

    Tuesday, October 19, 2010 3:20 PM
  • All ,

    I have had the same issue and want to share my solution.

    After a clean shutdown/reboot of the Hyper-v host , due to maintenance of multipath software, I got all the VM disappeared .

    After some troubleshooting I found that the LUN residing on SAN ,with multipath , had been set to "readonly" by Windows .

    So I cleared the readonly attribute within diskpart

    select disk x

    attr clean readonly

    HTH

    Stefano Colombo

    • Proposed as answer by Stefano_C Friday, April 06, 2012 10:13 AM
    Friday, April 06, 2012 10:13 AM
  • Hi All,

    Also wanted to add our experience as well. In the end for us the disappearing VM problem turned out to be corrupted configuration xml files for the VM's. Specifically a single missing ">" in the last line i.e. "</configuration" instead of "</configuration>"

    We shut down the Hyper V service, opened the xml file in notepad and added the missing bracket and saved the file, then restarted the base machine. All VM's were online when the restart was complete.

    HTH

    Ian Carson

    • Proposed as answer by Ian Carson Wednesday, July 25, 2012 1:23 AM
    Wednesday, July 25, 2012 1:14 AM
  • I just spent 2 hours figuring this out and thought it might help someone.

    In our case the configuration file was flawless with the proper </configuration> ending and no extra or missing char. It got me puzzled as what is wrong. sym links are there, files are good.....By sheer chance, we hit ctrl + S on notepad while viewing the XML file and it saved the file.

    Guess what, we then do a net start "Hype-v virtual Machine management" and there they are, machines are back. We just cycle through saving all the 5 XMLs one by one and all of them are back. It scared the hack out of me having them disappeared.

    So the steps are (for us it worked, please backup and test!):

    net stop "Hype-v virtual Machine management"

    nav to "%systemdrive%\ ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\"

    open up the XML in notepad.

    Don't touch the content (if they look intact with no errors) and just Save the file in notepad.

    net start "Hype-v virtual Machine management"

    Hope this helps :)

    Thursday, March 07, 2013 3:00 AM
  • Hi Martin,

    I had use the options 3 which is same as provide by Vincent Hu, and it work to for me to get back all the missing VM.

    I do encounter the same error when applying Icacls command then I realise the mistake of didn't change the folder name of my VM location, after correct the VM folder name in the command, no more error.

    Friday, April 19, 2013 2:37 AM
  • Opening the files in notepad and saving the xml files worked flawlessly for me. I was able to get 5 VMs back in no time. PS - the answer above is missing the r after net stop "Hype'r'

    Thank you



    Wednesday, July 17, 2013 8:24 PM
  • I just want to say thank you for this post.

    It worked flawlessly.

    We had a power outage and even though I was on APC backup power and shut them down they did not go down gracefully.

    I got the core systems for the weekend up and did not worry about the VM's till this morning "Monday"

    Well reading all the posts I decided to try yours first since it appeared to be the simplest of solutions and BAM it worked. I selected shut down of all 3 services from the management console, saved the .xml file in notepad as you stated. Then returned to the mmc and restarted all 3 services and there my accounting server was.

    I cannot tell you how good my Monday is now especially after thinking this was going to be a Monday for the ages.

    Monday, April 14, 2014 4:34 PM
  • Same problem thanks for this solution Ian Carson

    If you just double click the GUID.xml and in the browser if it looks like a bunch of text instead of an XML then check the </configuration> field at the end of the file, ours had </configuration>>

    Glad there is a simple solution to what seems a flaky system :-~


    Vista Ultimate Dell XPS M1330

    Monday, September 01, 2014 7:52 AM
  • All,

    I had the same problem after an MultiPath Io installation and Reboot of my host (Under Windows 2012 R2).

    The problem was the read only option.

    The right command under diskpart is:

    Attributes disk clear readonly

    I have to restart VMM management service and All Vm were back.

    Thank you for all informations.

    JR

    • Proposed as answer by eFloh Thursday, November 05, 2015 8:50 AM
    Saturday, February 28, 2015 5:45 PM
  • This is great and helped us immediately.

    You can check the readonly state in disk manager under control center\administration.

    the syntax in your post is broken, correct diskpart command is

    attr disk clear readonly

    Thursday, November 05, 2015 8:49 AM
  • In my situation: W2012R2 on a stand-alone server running Hyper-V, the problem turned out to be a hard-link in "C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines" that had no linked-file anymore. Starting Hyper-V manager resulted in vm's showing up, until a hard-link failed and the remaining vm's where not shown.

    Now, this happened without *any* events being logged or warnings/errors shown.

    The problem: a hard-link .xml file without the linked-file must have happened somewhere in de past: this machine is my test-server and I accept that the missing linked-file is somehow my own doing. But the problem, missing vm's in Hyper-V Manager, did not occur until I installed Visual Studio 2015 last Saturday.

    I solved the problem by:

    1. net stop "Hyper-v virtual Machine management"
    2. open "C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines" in File Explorer
    3. check for a missing linked-file: in File Explorer right-click the hard-link and select "Open file location". If the linked-file does not exist, you'll get an error to that affect
    4. If the linked-file does not exist, delete the hard-link to the nonexisting file

    When done, start Hyper-v virtual Machine management: net start "Hyper-v virtual Machine management". Start Hyper-V Manager to show your vm's

    Hope this is of some use to someone.

    Paul


    PE




    Monday, February 15, 2016 7:00 PM
  • Thank you-this saved my bacon!!!
    Thursday, March 16, 2017 9:54 PM