none
Heartbeat VM's - Cluster resource failed

    Question

  • Hi All,

    I have a Hyper-V cluster (Failover Cluster + SCVMM) with unexpected VM reboots.

    OS: Server 2012R2
    SCVMM: 2012R2 upgraded to 2016

    Event:

    Source: Cluster ServiceObjectName : 
    ObjectPath : Servername
    Monitor Alert: False
    Created: 7-3-2018 22:42:49
    --------------------------------
    Alert Description:
    Cluster resource 'SCVMM Servername of type 'Virtual Machine' in clustered role 'SCVMM Servername Resources' failed.

    Description:

    The unexpected shutdown occurs on vm's with heartbeat enabled. The VM unexpected reboots occur randomly on the entire environment but usually at night. It does not seem to come through commvault (our backup solution).

    It happens on SCVMM clusters with 10+ nodes and WAP (VM Management). 

    • Edited by TomvBr Saturday, March 10, 2018 12:26 PM
    Friday, March 09, 2018 3:13 PM

All replies

  • Hello,

    Basically, VM rebooted unexpectedly either due to  a problem with the Hyper-V host, or due to a problem with a VM guest OS.

    For the Hyper-V host, I would recommend to check the hardware resources, and make sure there are enough resources, such as CPU, Memory and Disk storage for the VMs. 

    For the VM, I would recommend to view the event logs on the Guest OS, and check out if there are any errors with the unexpected shutdown.

    Best regards,

    Andy Liu


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Monday, March 12, 2018 6:11 AM
  • Hello,

    Basically, VM rebooted unexpectedly either due to  a problem with the Hyper-V host, or due to a problem with a VM guest OS.

    For the Hyper-V host, I would recommend to check the hardware resources, and make sure there are enough resources, such as CPU, Memory and Disk storage for the VMs. 

    For the VM, I would recommend to view the event logs on the Guest OS, and check out if there are any errors with the unexpected shutdown.

    Best regards,

    Andy Liu


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thank you Andy,

    We use BL460c Gen9 XXLarge blades in combination with 3PAR's. (Storage Providers)
    There are enough resources for the VM's.

    Events in the VM's:

    The previous system shutdown at 02:06:44 on ‎12-‎3-‎2018 was unexpected. - Event ID 6008
    The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. - Event ID 41

    Storage Providers:

    Some storage providers are random locked in SCVMM. We have to unlock them everyday.. Maybe related?

    ISSUE
    When try to delete storage array Unable to acquire a Delete lock on object XX of type StorageArray because it is locked by task 00000000-0000-0000-0000-000000000000 (Unknown) with a NoDelete lock.

    IMPACT
    C

     RESOLUTION
    Delete the lock on VMM sql database
    select * from tbl_VMM_Lock 
    exec prc_vmm_releasealllocks

    I will start debug logging.

    Edit:
    I have found the following topic, seems similar. But not a solution in our case.

    http://www.edugeek.net/forums/windows-server-2012/161420-server-2012r2-hyperv-vms-randomly-reboot.html


    • Edited by TomvBr Monday, March 12, 2018 9:34 AM
    Monday, March 12, 2018 9:15 AM
  • Hello,

    Is there UPS working with the hosts?

    According to the following KB article, it may cause the VM from shutting down unexpectedly.

    https://support.microsoft.com/en-nz/help/3058418/vm-shutdown-actions-don-t-run-when-a-host-shuts-down-because-of-a-low


    Best regards,

    Andy Liu


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, March 13, 2018 8:06 AM
  • Hello,

    Is there UPS working with the hosts?

    According to the following KB article, it may cause the VM from shutting down unexpectedly.

    https://support.microsoft.com/en-nz/help/3058418/vm-shutdown-actions-don-t-run-when-a-host-shuts-down-because-of-a-low


    Best regards,

    Andy Liu


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    There is no UPS.

    At this moment I am testing several software & hardware components. I will post the updates here.

    • Edited by TomvBr Friday, March 16, 2018 11:08 AM
    Friday, March 16, 2018 11:07 AM