none
DPM Backups of Hyper-V guests on shared SMB storage fails with 0x80004005 RRS feed

  • Question

  • Scenario:

    Hyper-V 2012 Hosts, DPM 2012, guests are a mixture of 2008 R2 & 2012. Backend SMB storage server is running windows server 2012. Oddly, if you run a consistency  check it will successfully fix the problem and take a snapshot, but the next time backups run it'll fail all over again. I've tried removing/readding them to the protection group and that doesn't do anything. I've bounced between different SMB storage servers, different hosts, doesn't matter.


    Recovery point backups fail with this error in DPM console:

    Affected area:    \Backup Using Child Partition Snapshot\MACHINE
    Occurred since:    4/23/2014 1:00:48 PM
    Description:    Recovery point creation jobs for Microsoft Hyper-V \Backup Using Child Partition Snapshot\MACHINE on MACHINE have been failing. The number of failed recovery point creation jobs = 1.
     If the data source protected has some dependent data sources (like a SharePoint Farm), then click on the Error Details to view the list of dependent data sources for which recovery point creation failed. (ID 3114)
        DPM encountered an error while performing an operation for \\SMBSERVER\HYPERVSTORAGE@{4DC41554-E22D-4A4D-A51C-3CB1E1C4F79C}\VM1\MACHINE\Virtual Machines\ on HOST (ID 2033 Details: The parameter is incorrect (0x80070057))
        More information
    Recommended action:    1) Refer to the detailed error code in the description above. Retry this operation after the issue has been fixed.
    2) If the error was due to insufficient resources, then it could be a transient failure and you should retry this operation after some time.
        Create a recovery point...
    Resolution:    To dismiss the alert, click below
        Inactivate

    This is logged on the Hyper-V host in Hyper-V-VMMS Admin log:

    VSS writers inside virtual machine 'MACHINE' failed to perform BackupComplete corresponding to shadow copy (VSS snapshot) set with id '59EFCF44-A7E7-4D9F-8D56-5092501BC8F4': Unspecified error (0x80004005). (Virtual machine ID 7DB8E8F9-A2F0-4E0E-9F82-9CA15A38EE62)



    Wednesday, April 23, 2014 8:23 PM

Answers

  • In case someone runs into this again it turns out this was a combination of things. For one, I had SCVMM live migration protection setup and somehow my DPM computer account got removed from the permissions on SCVMM. So that was part of the issue.

    The second issue I found by looking in C:\Program Files\Microsoft System Center 2012\DPM\DPM\Temp and checking the various logs. I noticed this line when doing testing of the SMB backups:

        StartDRDataTransferBlock.cs(159)        1521B49A-6B53-4399-A6FE-86BB59299A56    NORMAL    SendRAReadDatasetDelta Start Bitmap ID = 8095dec5-f42f-44e0-b9d9-94f9652908d6, End Bitmap ID = 9162c061-0006-47ff-94d8-72b5ce1052ce, Volume = \\SMBSTORAGESERVER\HyperVStorage\*\\?\Volume{07f88ae8-6e16-4848-b061-0cc80b976c25}\*SMBSTORAGESERVER

        StartDRDataTransferBlock.cs(179)        1521B49A-6B53-4399-A6FE-86BB59299A56    NORMAL    Adding PolicyExclusion for \\SMBSTORAGESERVER\HyperVStorage\*\\?\Volume{07f88ae8-6e16-4848-b061-0cc80b976c25}\*SMBSTORAGESERVER, Specification = exclusion*.vhd, IsRecursive = True

    For some reason it was matching ALL vhds stored on that server to the exclusion pattern set to exclude hyper-v page file VHDS. No idea why, but when I removed it the backups successfully started again.
    Wednesday, May 7, 2014 5:48 PM

All replies

  • Hi just checking you have installed DPM agent on all Hyper-V hosts and SMB Hosts.  and that you have Delegation rights setup on the SMB hosts in AD, and the share security has the AD computer objects for the hyper-V hosts ?

    have you also installed SP1 on DPM 2012 ?

    regards

    Mark


    Friday, April 25, 2014 11:52 AM
  • DPM Agent is installed on all hosts and the SMB Hosts. Share security is everyone full control, the NTFS in said share are all the Hyper-V hosts & SMB servers with full control

    However, I wasn't aware that the SMB hosts needed delegation rights. Do I need to just grant them trust for any service?

    e: SP1 is installed, and i'm up to the latest update before the one that JUST came out a few days ago.

    Friday, April 25, 2014 4:36 PM
  • Hi yes. set trust for any service . then you will need to reboot.

    let us know how you get on !

    Cheers

    Mark

    Monday, April 28, 2014 8:54 AM
  • On the host computer I enabled trust this computer for any service and restarted after letting AD replicate. Attempted to take a backup of the virtual guest and it failed, and like before a consistency check works just fine and creates a valid recovery point just the actual recovery point process will not.

    It does a log a different error than before:

    Hyper-V VMMS Admin log

    Event id 10170

    Requester reported unsuccessful backup for the virtual machine 'GUEST'. (Virtual machine ID D34BCADF-1571-47F3-9E6A-2E4F937F6E02)

    Also event ID 16010

    The operation failed.



    Monday, April 28, 2014 7:47 PM
  • In case someone runs into this again it turns out this was a combination of things. For one, I had SCVMM live migration protection setup and somehow my DPM computer account got removed from the permissions on SCVMM. So that was part of the issue.

    The second issue I found by looking in C:\Program Files\Microsoft System Center 2012\DPM\DPM\Temp and checking the various logs. I noticed this line when doing testing of the SMB backups:

        StartDRDataTransferBlock.cs(159)        1521B49A-6B53-4399-A6FE-86BB59299A56    NORMAL    SendRAReadDatasetDelta Start Bitmap ID = 8095dec5-f42f-44e0-b9d9-94f9652908d6, End Bitmap ID = 9162c061-0006-47ff-94d8-72b5ce1052ce, Volume = \\SMBSTORAGESERVER\HyperVStorage\*\\?\Volume{07f88ae8-6e16-4848-b061-0cc80b976c25}\*SMBSTORAGESERVER

        StartDRDataTransferBlock.cs(179)        1521B49A-6B53-4399-A6FE-86BB59299A56    NORMAL    Adding PolicyExclusion for \\SMBSTORAGESERVER\HyperVStorage\*\\?\Volume{07f88ae8-6e16-4848-b061-0cc80b976c25}\*SMBSTORAGESERVER, Specification = exclusion*.vhd, IsRecursive = True

    For some reason it was matching ALL vhds stored on that server to the exclusion pattern set to exclude hyper-v page file VHDS. No idea why, but when I removed it the backups successfully started again.
    Wednesday, May 7, 2014 5:48 PM