none
Hyper-V host locked up RRS feed

  • Question

  • Server 2016 Hypver-V host locked up. 

    Logged in, black screen. Unable to access task manager.

    2 running virtual machines. 1 running Exchange 2016, one is a DC.

    I can RDP into Exchange so that can be properly shut down. It too has windows updates pending. The other virtual machine is pingable.

    Windows update on host is set to automatically download updates but set to manually install. The win update service was not running but is currently starting. 

    Volume Shadow copy service  on host is currently locked on stopping.

    I can access the host's services and files remotely, just the desktop appears to be locked.

    I am away from the office so I can only access the host via RDP at the moment. I'll be there in person on Monday. 

    What is the best course of action to reduce the chance of corrupting the host? Is there a way to open the cmd at the lock screen to attempt a shutdown /r ?

    Thanks

    Thursday, February 13, 2020 11:11 PM

All replies

  • Hi,

    If the Hyper-V host is a physical server, you may access it via the hardware vendor’s remote management, like iLO/iDRAC depending on the server hardware, from there you may try to log in with the console.

    If the Remote Desktop is black and does not react to any commands/actions, then you could try using Remote PowerShell to gain access to your Hyper-V host, if you can access it with PowerShell you may then be able to cleanily restart the host.

    https://docs.microsoft.com/en-us/powershell/scripting/learn/remoting/running-remote-commands?view=powershell-7

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, February 13, 2020 11:38 PM
  • Thanks for the reply.

    I'm not sure if the host is setup for remote management via powershell. What cmdlet would it be to try and access the remote hyper-v management console?

    I have a 2nd Hyper-V server on the network but not set a s a cluster (hardware can barely handle what they're running so clustering wouldn't be practical). I tried adding the frozen one to the healthy one's management console via the connect to another computer option but nothing happened.

    How about using this cmd from another server logged in as the admin: shutdown /m \\10.10.10.10 /f /r

    Random IP representing the locked Hyper-V host.

    Thursday, February 13, 2020 11:47 PM
  • The PowerShell remote cmdlets are found in my previous link.

    To restart a server remotely via Command Prompt:

    shutdown -r -m \\RemoteServer -t -01 
    If you add the /F switch it will close all running applications on the remote computer.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, February 13, 2020 11:51 PM
  • I have a 2nd remote software that allows for ctrl alt del input and tried logging in as a secondary user. The remote software then quit working after the 2nd user was still logging in.

    I was able to restart the service for that software remotely and got it working again.

    Logged back in as the admin and the desktop came back up.

    In event viewer, the only thing I can see that might have caused the issue is VSS errors:

    The activation of the CLSID {************} timed out waiting for the service VSS to stop. Error code 10029, source dcom.

    Any thoughts on how to fix this error? Not really finding anything.

    The host also runs my backup software.

    Saturday, February 15, 2020 3:29 PM
  • Most VSS related issues are backup related, depending on the backup software you're using you might want to reach out to their respective forums/support.

    Blog: https://thesystemcenterblog.com LinkedIn:

    Saturday, February 15, 2020 3:38 PM
  • As what others said it could be backup related, try disabling the backup service. And try to duplicate the issue, if you can't duplicate the issue or it's working normally then you can narrow down the issue to the backup software.

    However, after you disable the backup service and when you try to duplicate the issue and still exists.

    Then it could be something else. Or try updating the host server (graphics update if any), or if the backup software has updates try installing it as well.

    Check if the system is running out of memory when the issue is happening, you might need to increase RAM.

    Good luck!


    Every second counts..make use of it. Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Monday, February 17, 2020 5:16 AM
  • Hi Susan_773,

    Thanks for your post! 

    I list some troubleshooting directions below to help us scope the issue, for your reference.

    1. When the issue occurs on the hyper v host, are we doing backup on the host?

    2. If yes, what the backup tool do we use, windows server backup or third-party backup tool?

    3. If we are running windows server backup, then please try to backup the machine again at the maintenance time to check if the issue will be reproduced?

    4. Please open cmd, run vssadmin list writers, check if there's any error of the vss writers? If yes, let's go into fix vss writer issue topic.

    5. If vss writers are well, while the issue can be reproduced when backup the host using win backup:

        a. please check if there's any anti-virus application running when we doing backup.

        b. please check if the hyper V host has installed the latest windows update, if not, please install it.

    6. If the issue can be reproduced while unable to be resolved by general solutions, then it may need to capture VSS dump to deep troubleshoot, however, please understand dump analysis is out of forum supporters' scope. 

    https://support.microsoft.com/en-us/help/4051701/global-customer-service-phone-numbers

    7. If the issue is random and not reoccur, then it's really hard to find the root cause of the issue.

    8. If the issue is not backup related, while we can reproduce the issue, then it's recommended to troubleshoot the issue via system event errors, and system performance. For system performance, we may direct to another topic.

    Thanks for your time! 

    Best Regards,

    Anne


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Monday, February 17, 2020 8:45 AM
    Moderator
  • I was successfully able to reboot the host. I was able to get a cmd open and do a shutdown /f /r.

    There was a stuck backup job on the host.

    Veritas Backup exec 20

    vssadmin list writers gave no errors after the reboot.

    The host does have windows updates pending but it has been running backups with the current patches for awhile now.

    Running a backup now and will see what happens.

    Wednesday, February 19, 2020 4:53 AM
  • Backup got stuck again.

    While it is stuck, the vssadmin list writers hangs.

    Wednesday, February 19, 2020 6:11 PM
  • You might need to consult with Veritas as it seems that's the root cause of your issues here, you can ask them in their forums over here: https://vox.veritas.com/t5/Backup-Exec/bd-p/backup-exec

    Here's also a Veritas BackupExec administrator's guide that you might want to take a closer look at:

    Veritas Backup Exec Administrator's Guide
    https://www.veritas.com/support/en_US/doc/59226269-99535599-0/v53950404-99535599


    Blog: https://thesystemcenterblog.com LinkedIn:

    • Proposed as answer by Leon Laude Monday, February 24, 2020 6:29 AM
    Wednesday, February 19, 2020 6:42 PM
  • Hi Susan_773,

    Thanks for your feedback.

    According to your information, it seems we can reproduce the issue via backup the machine with the third-party tool.

    1. To further rule out the issue, you may try if the issue still exists when we use windows backup tool?

    2. If the issue is not with windows backup tool, then it seems the issue is indeed related with the third-party backup tool.

    3. Try to uninstall anti-virus application when backup, check if it could work. If it's still not work, it's recommended to consult the third-party backup tool supporters.

    Thanks for your time! If you have any other concerns, please feel free to let me know.

    Best Regards,

    Anne


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, February 20, 2020 6:07 AM
    Moderator
  • Hi Susan_773,

    Greetings!

    I am writing to check if you have any other concerns or questions on the issue? If yes, please feel free to feedback. If not, would you please mark the useful reply as answer to end this thread.

    Thanks for your time!

    Best Regards,

    Anne


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Monday, February 24, 2020 5:45 AM
    Moderator
  • After several reboots and shrinking down a virtual hard drive, the problem seems to have resolved itself.
    Thursday, March 12, 2020 5:41 PM
  • Great to hear that!

    (Please don't forget to mark helpful replies as answer)


    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, March 12, 2020 5:47 PM