none
DPM 2019 - Impossible to recover VMs from tape when colocation is enabled RRS feed

  • Question

  • Hello everyone,

    I'm testing DPM 2019 and am facing an issue when I want to recover Hyper-V VMs from a backup to tape I made with colocation of 2 protection groups.

    Here is my env:

    - This is a physical host with Windows Server 2019, Hyper-v 2019 and DPM 2019.
    - I created 2 protection groups. Each of them save 3 VMs (1x Server 2008 R2, 1x Server 2012, 1x Server 2019 for the VMs)
    - The physical server hosting the thing is in a test domain whereas all other guest OS inside the VMs are in a different domain.
    - No DPM agent has been installed on the VMs
    - The backup to tape doesn't raise any error.
    - When I want to recover a specific VM from PG1 it works. But when I want to recover a VM from PG2, the following error appears:
    "Event 3111 - The recovery jobs for Microsoft Hyper-V RCT\VMNAME that started [...] have completed. Most or all jobs failed to recover the requested data. (ID 3111)
    The protection agent on VMNAME.domaine.ch was temporarily unable to respond because it was in an unexpected state. (ID 60 Details: Internal error code: 0x809909B0)"

    In the system log on physical host, we can see Event 7034 with "The DPMRA service terminated unepectedly. It has done this x time(s)".

    In the Application logs on physical host, we have Event 1000 "Faulting application name: DPMRA.exe, version: 10.19.58.0 [...]"

    When I check the log in Program Files\Microsoft System Center\DPM\DPM\Temp\DPMRACurr.errlog.Crash, I can see various things like: "WARNING Failed: Hr: = [0x80070002] : Encountered Failure: : lVal : hr"
    At the very end of that file, I can notice: "CRITICAL Raising Watson for process".

    Browsing the web didn't help me a lot about these error codes. I can assure there is no network issue. This is a test environment with almost no traffic and it is not overloaded.

    I tried to disabled Windows FW / AV but the same happenened.

    What's weird is that it starts recovering very few data (like 40 MB) before it fails.

    Does anyone have an idea on how to resolve that ?

    Your help is already appreciated.


    • Edited by dd-cges Friday, July 5, 2019 3:17 PM Added a info
    Friday, July 5, 2019 3:09 PM

All replies

  • Hello,

    I can notice now that the backup to tape of a single protection group without protection group has the same behaviour than what I described: backup to tape seems to work ok but the recovery has the same symptoms.

    I'd really appreaciate some help and answers.

    Thanks by advance to all of you who will give some support.

    Tuesday, July 9, 2019 7:16 AM
  • Hello,

    So you have a DT (Disk-to-Tape) configured protection group?

    How are you trying to recover the Hyper-V virtual machine? Are you recovering the whole virtual machine or just the virtual machine files?

    Could you provide some more information of how you configured your protection groups?

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:

    Tuesday, July 9, 2019 8:10 AM
  • Hello Leon,

    Sorry for the late answer, urgent and unexpected things required my attention.

    To answer your question: I'm trying to recover the whole VM (and so its files) (RCT\vmName).

    I did more test and here are things I can add to the first description of the issue:

    - When I combine B2T with B2D (Backup to tape with Backup to disk), things are ok; I can recover from the tape.
      - If I then remove the protection group (PG) by ticking "Delete Protected data" + "Delete data source replicas from disk" AND I don't tick "Expire all the datasets in the tapes. Tapes will be marked as free when they are online"; then I can still recover VMs from the tape AS LONG AS I mark the tape as not free before. It makes the "View Content" visible and allow me to recover.

      - If I then remove the rest that became inactive PG, then I cannot recover anything from the tape. I don't have the VIew content button neither.

    - When I do a simple B2T without B2D, nothing is recoverable.

    Now, I could notice the following errors:

    - Eventvwr: Event 1001, Windows Error Reporting: Fault bucket, type 0. Event Name: APPCRASH. Problem signature: P1: DPMRA.exe [...]
    - In the scenario where I only do a B2T of 2 VMs, I can see in "All jobs" a failed job:
        Verification failed for the tape backup of RCT\VMNAME2 on SERVER.COM because the tape backup failed (ID 30158). The other VM has no error of that kind.

    I forgot to mention the DPM 2019 version I'm using in my test env: 10.19.58.0.
    SQL server is 2017.
    SSRS is 14 and SSMS is 18.0

    I already appreciate all help you all can provide. Thank you by advance.

    Friday, July 19, 2019 2:47 PM
  • Hello again,

    I'm continuing to investigate this case as we cannot put that in production if it doesn't work.

    I've discovered that if I try to do a Backup to Tape (only) of a VM that has configuration version 5.0: it works.

    "Only" VMs with configuration version 9.0 and RCT fails to recover with errors I mentioned.

    Anyone has any ideas ? I'd be nice if Microsoft could better describe the error.

    Wednesday, July 24, 2019 1:43 PM
  • Your issue appears to be what Mike tells about in the following thread:
    DPM 2019 - Is it possible to choose not to use RCT but Child Partition ?

    Blog: https://thesystemcenterblog.com LinkedIn:

    • Proposed as answer by Leon Laude Tuesday, August 20, 2019 7:06 AM
    Wednesday, August 7, 2019 12:29 PM