none
DPM Replica is full - CANNOT Modify Replica RRS feed

  • Question

  • Hi Everyone

    We recently had our DPM upgraded from 2016 to 2019.

    And all has been working fine until now. I have certain backups that are no longer backing up with error: 

    Affected area: *Affected Server*
    Occurred since: 29/11/2019 7:10:32 AM
    Description: The replica of Microsoft Hyper-V *Affected Server* on *Affected Server*.CLUSTER.Contoso.com is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. You can recover data from existing recovery points, but new recovery points cannot be created until the replica is consistent. 

    For SharePoint farm, recovery points will continue getting created with the databases that are consistent. To backup inconsistent databases, run a consistency check on the farm. (ID 3106)
    DPM is out of disk space for the replica. (ID 58 Details: There is not enough space on the disk (0x80070070))
    More information
    Recommended action: Ensure there are no pending disk threshold alerts active for this data source and rerun the job. DPM may have automatically grown the volume and resolved the alerts.
    Synchronize with consistency check.
    Modify Disk Allocation ...
    Run a synchronization job with consistency check...
    Resolution: To dismiss the alert, click below
    Inactivate

    ========================================================

    I have browsed around and tried ps scripts to modify the Replica size. or to enable the Greyed out "Modify Disk allocation" option. NO LUCK. 

    I read up somewhere about legacy backups and modern backups? but still cant grasp how its related to my issue and resolve it.

    Im leaning on deleting all protection groups and starting from scratch? (lowering my backup days scheduled)

    But question on that is, Will that clean out my "replica" folder that is full (The replica folder i understand are volume drives, located in your program files/dpm/dpm/replica)???

    Tuesday, December 3, 2019 1:47 PM

All replies

  • Hello,

    I've seen this issue quite a lot but no real solution as of today, a fix might be coming in the future.

    Is this a host-level or guest-level backup?

    You could try removing the protected workload from the protection group and then re-add it (with retain data).

    If that doesn't work, you can try checking the DPM log for more clues:

    • C:\Program Files\Microsoft System Center\DPM\DPM\Temp\MSDPMCurr.errlog

    If you have the possibility to delete old recovery points, then simply remove the workloads from the protection group (with the "delete data" option), this will delete the actual backu data from the DPM storage pool.

    The replica folder that you see under "C:\Program Files\Microsoft System Center\DPM\DPM\Volumes\Replica", is really a volume mount point pointing to the replica on the DPM Storage pool disk.

    Example:

    C:\Program Files\Microsoft System Center\DPM\DPM\Volumes\Replica\40241b1c-17ac-499a-a82c-6a0722ab8b16\Full\C-Vol\"

    The bolded folder is only a mount point, there is will be no space consumed on C: drive.

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:

    Tuesday, December 3, 2019 2:14 PM
  • I also experienced this with DPM 2019 that was writing to a SAN that had two volumes. In the past on previous versions of DPM, I never paid much attention to where the data was being written. DPM balanced it out as needed. However, this time, DPM completely filled one of the volumes and was basically ignoring the other volume that was 50 percent free. My solution was to move the backups to some of my other DPM servers, wait for the backups to expire, then remove the volumes in the SAN and create just one volume. I moved the backups back to the server and now DPM doesn't have a choice as to which volume to fill up. 
    Tuesday, December 3, 2019 4:56 PM
  • Hi Leon 

    Thanks for the feedback

    I've checked the logs, nothing that i can see that might be able to help with the current issue, but ill keep checking them.

    We do Host-level Backups

    So am i right by saying that if i delete old recovery points (stop protection of member server) & "delete Replica on disk" this should automatically free up space on our replica?.

    If this is true. I might just do that and drop my retention range from 30 days to 20 days. I have a feeling that 30 days is too much for our replica.

    This never was the case until we upgraded to 2019.. quite disappointing actually.

    Wednesday, December 4, 2019 8:38 AM
  • So am i right by saying that if i delete old recovery points (stop protection of member server) & "delete Replica on disk" this should automatically free up space on our replica?.

    >> Yes that's correct.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Wednesday, December 4, 2019 8:47 AM
  • Thanks

    Ill have a bash and revert back the next day.

    Fingers crossed.

    Wednesday, December 4, 2019 9:17 AM
  • Ok so 50/50

    So i decided to create a New protection group with a retention range of 5 days.

    Added two servers on the in the beginning. fine cool. works.

    this morning i checked up, all fine still. 

    next thing i did is remove a member from a previous protection group (20days retention range) and delete its replica.

    once done, i tried to add it to my new protection group (5 days). and i end up with the same issue. "replica is full".. 

    strange. 

    on another note: the server i removed from the 20 days group and deleted its replica.

    I was randomly browsing my dpm C:\ drive and found on the root of my c drive a - "C:\DPMVolumes\Microsoft Hyper-V VSS Writer\Volume drive of the Server i just removed" 

    Weird that its located there??? and also. i removed it on dpm? so why is this volume still there and accessible?.

    Can i delete this volume under disk management?


    Thursday, December 5, 2019 7:39 AM
  • Is this a host-level or guest-level backup of the Hyper-V virtual machines?

    When you add a new data source to an existing or new protection group, DPM makes two new volumes, adds VSS shadowstorage - creates an initial replica, then makes a VSS shadow copy.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, December 5, 2019 9:02 AM