none
DPM2010 single 2K3 SP2 x64 VM guest backup fails on CSV, VSS error RRS feed

  • Question

  • Hi,

    We have a CSV with 3 nodes and 14 guests (a mix of 2K3 R2 SP2 x64 and 2K8R2 SP1) spread over the nodes. All nodes are Windows 2008 R2 SP1.

    Since about a week a single 2K3 guest backup has started failing and we are unable to get it to back up properly again. No updates or new programs have been installed on the affected guest or any of the CSV nodes before it started failing. SQL2005 SP4 is installed on this guest. The SQL db's are themselves backed up in a different protection group that has no issues whatsoever.

    When we try to run the sync job with consistency check it shows the following behavior:

    - DPM starts the job

    - The host node CSV goes into redirected mode as it should and returns to Online as the backup job fails after a few moments

    - At the same time in the Hyper-V manager we can see the status for the affected guest change to "Creating VSS Snapshot set" which after a few seconds changes to "Creating VSS Snapshot set - failed"

    - The DPM job stops and reports the inconsistent replica state

    We have already tried the following:

    - removing the affected guest from the protection group and re-adding it

    - rebooting the node the guest was running on and trying the sync job with consistency check

    - rebooting the guest and then trying again

    - moving the guest to another node and trying again

    - restarting the affected services (Hyper-V Virtual Machine Management) to clear the VSS error and then trying again

    - increased the storage size of the replica volume and then trying again

    none with any positive outcome.

    The reported error in DPM is:

    Affected area:    \Backup Using Child Partition Snapshot\GuestWithProblem
    Occurred since:    22/08/2012 9:03:03
    Description:    The replica of Microsoft Hyper-V \Backup Using Child Partition Snapshot\GuestWithProblem on GuestWithProblem.HPRVCLSTR01.domain.com is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. You can recover data from existing recovery points, but new recovery points cannot be created until the replica is consistent.

    For SharePoint farm, recovery points will continue getting created with the databases that are consistent. To backup inconsistent databases, run a consistency check on the farm. (ID 3106)
        DPM encountered a retryable VSS error. (ID 30112 Details: VssError:The writer experienced a transient error.  If the backup process is retried,
    the error may not reoccur.
     (0x800423F3))
        More information
    Recommended action:    Check the Application Event Log on hostnode.domain.com for the cause of the failure. Fix the cause and retry the operation.
    For more information on this error, go to http://go.microsoft.com/fwlink/?LinkId=132612.
        Synchronize with consistency check.
        Run a synchronization job with consistency check...
    Resolution:    To dismiss the alert, click below
        Inactivate alert

    In the Application Eventlog on the host we can see EventID 8224 "The VSS service is shutting down due to idle timeout. "

    In the Hyper-V VMMS Eventlog on the host we see EventID 10102 "Failed to create the backup of virtual machine 'GuestWithProblem'. (Virtual machine ID )"

    When we run the vssadmin list writers command on the host node all writers are ok except this one:

    Writer name: 'Microsoft Hyper-V VSS Writer'
       Writer Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
       Writer Instance Id: {9f5e9c95-296b-44ea-a1b4-75e5a4053274}
       State: [1] Stable
       Last error: Retryable error

    The error is cleared when the Hyper-V Virtual Machine Management service or the node is restarted.

    On the guest there are no errors or warning in the eventlogs but we do see several writers in a failed state:

    Writer name: 'System Writer'
       Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
       Writer Instance Id: {486cd855-7bc3-4924-ba1a-4602e2fe95e6}
       State: [7] Failed
       Last error: No error

    Writer name: 'SqlServerWriter'
       Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
       Writer Instance Id: {012ba53e-a452-4bd7-8ecc-dea207cc09b0}
       State: [7] Failed
       Last error: No error

    Writer name: 'IIS Metabase Writer'
       Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
       Writer Instance Id: {729f2090-f5e1-4414-bd18-e344b04ef537}
       State: [7] Failed
       Last error: No error

    Writer name: 'WMI Writer'
       Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
       Writer Instance Id: {6ab65099-af57-4d12-8938-6b7a737a2a92}
       State: [7] Failed
       Last error: No error

    These states are cleared after a reboot but the backup still fails afterwards and causes the same writer issues.
    "vssadmin list shadows" returns no results.

    We're a bit stumped as to why this problem suddenly occurred after running perfectly for months.

    Does anyone have a suggestion as to what we could try next?


    • Edited by jco_fa Wednesday, August 22, 2012 9:52 AM
    Wednesday, August 22, 2012 9:50 AM

Answers

  • There is a KB article that provides some information about this error.  It is 2462424 - "A System Center Data Protection Manager 2010 initiated Hyper-V backup using a child partition for a guest fails with error 30112" at http://support.microsoft.com/kb/2462424.  Has the error returned at this point?

     --------------------------------------------------------------------------------
     Regards, Michael V [MSFT] - This posting is provided "AS IS" with no warranties, and confers no rights.

     
    Thursday, September 13, 2012 11:04 PM
    Moderator

All replies

  • no ideas?
    Monday, August 27, 2012 7:03 AM
  • okay, i don't know which stars aligned with which planets or what current tidal forces are at work or which direction curiosity's camera was pointing at the time but after trying the sync job with consistency check again it suddenly started doing a correct backup and created a valid recovery point.

    We haven't had a proper recovery point for the full hyper-v for 2 weeks and now it suddenly, magically works again without trying anything since the 22nd?

    Tuesday, August 28, 2012 7:55 AM
  • There is a KB article that provides some information about this error.  It is 2462424 - "A System Center Data Protection Manager 2010 initiated Hyper-V backup using a child partition for a guest fails with error 30112" at http://support.microsoft.com/kb/2462424.  Has the error returned at this point?

     --------------------------------------------------------------------------------
     Regards, Michael V [MSFT] - This posting is provided "AS IS" with no warranties, and confers no rights.

     
    Thursday, September 13, 2012 11:04 PM
    Moderator