DPM 2010 - backing up a Hyper V VM - consistency check fails RRS feed

  • Question

  • Hi, we have a 2 node Hyper V cluster that is being backed up using Child Partition Snapshot by DPM 2010. We have 1 particular VM that was backing up fine but now it is showing up as the Replica is inconsistent. I have run a Consistency Check and it gives the following error:

    The replica of Microsoft Hyper-V \Backup Using Child Partition Snapshot\SQL***** on SCVMM ****** ******************** is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. You can recover data from existing recovery points, but new recovery points cannot be created until the replica is consistent.

    An unexpected error occurred while the job was running. (ID 104 Details: The parameter is incorrect (0x80070057)).

    All other VM's on the same cluster are backing up fine.

    On the VM I get the following error:

    - <Eventxmlns="">
    - <System>
      <Provider Name="VDS Basic Provider"/>
      <TimeCreated SystemTime="2012-03-15T15:39:27.000000000Z"/>
      <Security />
    - <EventData>


    The VM is a SQL 2008 server whose databases are backing up fine by same DPM 2010 server.

    On the Hyper V host the 0x80070057.txt file from C:\Program Files\Microsoft Data Protection Manager\DPM\Temp> shows:

    0C34    1D40    03/14    19:31:32.996    31    aaxmlprocessingutils.cpp(49)        57091C2F-6142-432B-9ACC-88714860197B    WARNING    Failed: Hr: = [0x80070057] : F: lVal : E_INVALIDARG
    0C34    1A48    03/14    21:12:03.291    18    checksumprovider.cpp(221)    [0000000000773DE0]    57091C2F-6142-432B-9ACC-88714860197B    WARNING    Failed: Hr: = [0x80070057] : U: lVal : E_INVALIDARG
    0C34    1A48    03/14    21:12:03.291    18    filefixupreader.cpp(137)    [00000000039C94D0]    57091C2F-6142-432B-9ACC-88714860197B    WARNING    Failed: Hr: = [0x80070057] : F: lVal : m_pChecksumProvider->GetChecksums(&m_pbInChecksumBuffer, &m_dwInChecksumBufferSize)

    Thursday, March 15, 2012 4:36 PM

All replies

  • Have you checked the eventlog of the VM when the host level backup fails? it probably shows lots of errors regarding the backup of SQL server databases.
    Friday, March 16, 2012 4:08 PM
  • Hi Marcus, thanks for you reply, I have checked the Eventlog and the only error I get on the VM is the VDS one above, the databases back up fine, its the backup of the actual VM that is failing.
    Monday, March 19, 2012 12:31 PM
  • I had this problem with one of our VMs not being backed up. This guest was running out of space (I think it had only a few Kbs free), so I extended the C: drive and the backups returned to normal.

    Hope it helps

    Tuesday, March 20, 2012 12:25 PM
  • Hi Marcos, thanks for your reply. The Guest and host both have plenty of space on their C drives.
    Tuesday, March 20, 2012 12:44 PM
  • The Eventlog entry on the VM guest seems to be this and can be ignored. What do you think?

    Can you please check for the replica path on DPM for this data source and check if you can browse into the directory using Windows explorer? Maybe there is something wrong with the replica and/or recovery Point volume.

    Are you using colocation on disk for this PG?

    Tuesday, March 20, 2012 1:28 PM
  • The Eventlog entry on the VM guest seems to be this and can be ignored. What do you think? -yes, I saw that too, I wasnt sure if it was relevant or not

    Can you please check for the replica path on DPM for this data source and check if you can browse into the directory using Windows explorer? Maybe there is something wrong with the replica and/or recovery Point volume.

    I can browse so far in the replica path but it won’t let me open the volume (permission denied) c:\Program Files\Microsoft DPM\DPM\Volumes\Replica\Microsoft Hyper-V VSS Writer\vol_19b9e572-f824-4f5a-aeed-9664b371d8e5

    Are you using colocation on disk for this PG?

    no we are not using co-location as we are not using tapes, we are using Short-term protection on Disk

    Tuesday, March 20, 2012 2:25 PM
  • I would suggest you look into the Folder by giving yourself access.

    My question regarding colocation was about disk based colocation which is possible for Hyper-V data. Are you using that?

    Tuesday, March 20, 2012 3:10 PM
  • The directory is a mounted volume, does that make a difference? I have full access permissions to it but still wont let me access it.

    I dont think we are using colocation.

    I have checked on the Host box and have found the following errors in the DPMRACurr ERRLOG file

    1E6C    0C70    03/27    14:19:22.300    31    hypervwriterhelperplugin.cpp(197)            NORMAL    Component Info, bstrLogicalPath:, bstrComponentName:12156589-84EF-410A-86A5-E69D36813AA2, eComponentType:2
    1E6C    0C70    03/27    14:19:22.300    31    vsssnapshotrequestor.cpp(236)    [0000000000453710]        NORMAL    CVssSnapshotRequestor::AddComponentForSnapshot [0000000000453710]
    1E6C    0C70    03/27    14:19:22.316    31    vsssnapshotrequestor.cpp(347)    [0000000000453710]        NORMAL    CVssSnapshotRequestor::AddVolumeForSnapshot [0000000000453710]
    1E6C    0C70    03/27    14:19:22.331    03    clusterutil.cpp(1757)            WARNING    ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    1E6C    0C70    03/27    14:19:22.331    03    clusterutil.cpp(1757)            WARNING    ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    1E6C    0C70    03/27    14:19:22.394    31    vsssnapshotrequestor.cpp(379)            NORMAL    ssLocalVolumeGuid = [\\?\Volume{5fa6910e-a163-40b7-b27a-f5ad444625c0}\], ssClusterVolGuid=[\\?\Volume{5fa6910e-a163-40b7-b27a-f5ad444625c0}\]

    Tuesday, March 27, 2012 2:31 PM
  • So you are saying that if you pull up the replica path of that datasource and try to open it in explorer you get an error? Have you checked in disk management of your Windows server if the disk maybe is of type RAW instead of NTFS? Any hints on if the disk is in shadow copy protection mode?
    Tuesday, March 27, 2012 3:02 PM
  • When I try to access the replica path in Explorer it gives me the 'You dont currently have permission to access this folder', when I click continue to get access it gives me the 'you have been denied permission to access this folder'. I check the Security tab and my user has full access.

    If I give myself permission to access to the disk in Disk Management, I can then access the replica path in Explorer.

    The disk is NTFS, I cannot see if shadow copy protection mode is on but I dont think so.

    Tuesday, March 27, 2012 3:33 PM
  • It is not OK that you cannot browse into that path. I think something is wrong with the disk. You could search for VolSnap errors in the eventlog maybe this gives you hints on if the volume is in shadow copy protection mode. There is a tool called vspmquery.exe to query if a volume is in shadow copy protection mode and another one vspmunset.exe to try to get it out of it.

    Another thing you can try: if you don't need to recovery points on disk for the given VM which fails to back up, then you shoudl try stop protection with removal of the disk based recovery points and then adding this VM again. This will remove the two disks and recreate them. If it was an issue with the replica and/or recovery disks, then this will "fix" it. But keep in mind you cannot restore the VM in question because you delete the recovery points by doing this. Before you try this you have to check if you are using colocation because if you delete the two volumes and you are using colocation, then you are going to delete recovery points for other machines as well...

    Tuesday, March 27, 2012 3:45 PM
  • I will try the first steps and see what we get.

    regarding the second paragraph, I have already tried removing the replica and recreating it and get the same error.

    Thanks for all the advice, I will get back to you.

    Tuesday, March 27, 2012 3:52 PM
  • You have to set permissions on the volume by going to the Disk Management section of the Server Manager.

    • Open the Server Manager
    • Under Storage, open Disk Management
    • Find the DPM volume who's name corresponds to the replica path for your protection group.
    • Right click on the volume and choose Properties. Go to the Security tab and give yourself full permissions.
    • If you see two DPM volumes with the same name, give yourself full access permission to both.

    You should now be able to browse the replica volume via Windows Explorer.

    Good luck!

    Friday, December 14, 2012 10:37 PM