none
Access Denied - Trying to protect a VM stored on SOFS

    Question

  • Hello,

    I am having problems trying to backup a VM, which is running on a Hyper-V Cluster and stored on a SOFS cluster (SMB).  This is a bran new DPM 2019 server install.  The DPM server is installed and successfully backing up file servers which have the DPM agent installed.  However, I am unable to backup VMs as a whole.  When I add the VM to a protection group, the initial replica fails.  And now, I also cannot perform a consistency check.  It fails with this error: 

    An unexpected error occurred while the job was running. (ID 104 Details: Access is denied (0x80070005))

    Here's what I have already done:

    • Installed DPM agent on all Hyper-V nodes in the Hyper-V cluster
    • Installed DPM agent on all SOFS nodes in the SOFS cluster
    • Installed the File Server VSS Agent Service on all SOFS nodes
    • Added the following computer accounts to the "Backup Operators" group on all SOFS nodes (DPM$, HYPERV_NODE1$, HYPERV_NODE2$, HYPERV_NODE3$ and HYPERV_CLUSTER$)
    • Gave all the computer accounts mentioned above, "Full Control" access to the file share and NTFS permissions on the folder tree, where the VM is stored on the SOFS.
    • After making all of these changes, I restarted all the SOFS nodes and the DPM server.

    I should mention that my new DPM server is running DPM2019, on Windows Server 2019, with SQL Server 2017.  My SOFS nodes and my Hyper-V nodes, are all running Windows Server 2016.

    Does anyone have any ideas?

    Tuesday, April 9, 2019 2:58 PM

Answers

All replies

  • Hello Marc,

    So you're using SMB storage right?

    Did you make sure you followed the following?

    • Turn on auto-mount on the server that is running Hyper-V to enable virtual machine protection.
    • Disable TCP Chimney Offload.
    • Ensure that all Hyper-V machine$ accounts have full permissions on the specific remote SMB file shares.
    • Ensure that the file path for all virtual machine components during recovery to alternate location is less than 260 characters. If not, recovery might succeed, but Hyper-V cannot mount the virtual machine.
    • The following scenarios are not supported:
      Deployments where some components of the virtual machine are on local volumes and some components are on remote volumes; an IPv4 or IPv6 address for storage location file server., and recovery of a virtual machine to a computer that uses remote SMB shares.
    • You'll need to enable the File Server VSS Agent service on each SMB server - Add it in Add roles and features > Select server roles > File and Storage Services > File Services > File Service > File Server VSS Agent Service.

    Reference:
    https://docs.microsoft.com/en-us/system-center/dpm/back-up-hyper-v-virtual-machines?view=sc-dpm-2019

    The cluster service account should have full permission on the specific Remote SMB file share.

    Note: Accounts should be explicitly added and not embedded in other accounts as odd behavior has been observed when using groups, this is not a requirement but it has been a tendency to cause issues.

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:

    Tuesday, April 9, 2019 4:01 PM
  • Hi Leon,

    Yes, I am using SMB storage.  I've seen and verified all of those points.  All except for the first one.  I am not sure what the "auto-mount" setting is on the Hyper-V servers.  When I search for Hyper-V and Auto-Mount, I only see results pertaining to automatic mounting of volumes.  But I fail to see why that is relevant here.  The Hyper-V nodes only have their OS volumes (C: drive) and nothing else.

    Other than that, I have done everything else.

    Tuesday, April 9, 2019 4:10 PM
  • You might have to dig through the DPM logs on the cluster nodes and the DPM server for more clues:

    DPM server:

    • C:\Program Files\Microsoft System Center\DPM\DPM\Temp\MSDPMCurr.errlog

    Protected servers:

    • C:\Program Files\Microsoft Data Protection Manager\DPM\Temp\DPMRACurr.errlog

    Blog: https://thesystemcenterblog.com LinkedIn:

    Tuesday, April 9, 2019 4:23 PM
  • Hi Leon,

    I've found this line in the DPMRAcurr.errlog file of the Hyper-V Node (on which the VM I am attempting to protect is currently running).  But I don't find it to be very helpful.

    3830 3B68 04/10 15:22:14.525 05 fsmtransition.cpp(111) [00000150D7FA19A0]  WARNING Failed: Hr: = [0x80070005] HasEventErrorCode: completion: 0xa10c, signature: 0xaabbcc00

    The very next line after that error code, is "CVssSnapshotRequestor::DoFailureCleanup".  So from here on, I'm assuming it is simply cleaning itself up following the error.

    One thing I did notice, I am able to protect VMs running newer versions of Windows Server, using RCT.  But most of my VMs are still running Windows Server 2012R2, and this issue seems to be only affecting my ability to protect those 2012R2 VMs.  I was successful in protecting two VMs running Windows Server 2016. Not sure if that helps to pin point the problem.

    Thanks for your continued help.

    Wednesday, April 10, 2019 4:38 PM
  • Can you compare the virtual machine configuration version of the non-working and the working VMs?

    For DPM to be able to use RCT, the VM configuration version must be 6.0 or higher.


    Blog: https://thesystemcenterblog.com LinkedIn:


    • Edited by Leon Laude Wednesday, April 10, 2019 6:25 PM
    Wednesday, April 10, 2019 6:21 PM
  • Correct.  The non working VMs are version 5.0.  The two that work, are version 8.0.  I realize that for RCT, you need version 6.0 or newer.  However, DPM server should still be able to protect these  VMs using non-RCT method.

    Wednesday, April 10, 2019 6:29 PM
  • Indeed DPM should be able to backup those, Windows Server 2016 has by default VM configuration version of 8.0.

    The documentation does state the following:

    The version of Integration Components that is running on the virtual machine should be the same as the version of Hyper-V on the server that is running Hyper-V.

    Reference:

    https://docs.microsoft.com/en-us/system-center/dpm/back-up-hyper-v-virtual-machines?view=sc-dpm-2019

    Could you try upgrading the VM configuration version of one VM? Upgrading has no effect on the workloads in the VM itself, but there is no way to go back to a previous configuration version.


    Blog: https://thesystemcenterblog.com LinkedIn:

    • Marked as answer by Marc Proulx Wednesday, April 10, 2019 7:00 PM
    Wednesday, April 10, 2019 6:43 PM
  • Thank you Leon.  That was the problem.  Once I upgrade the VM configuration to 8.0, the error went away.  Thank you for your help
    Wednesday, April 10, 2019 7:00 PM
  • I'm glad to hear that Marc, you're welcome!

    Blog: https://thesystemcenterblog.com LinkedIn:

    Wednesday, April 10, 2019 7:02 PM