none
Large Daily 2008 R2 System State or BMR Synchronization on Secondary DPM 2010 RRS feed

  • Question

  • We have a Primary/Secondary DPM 2010 configuration (Windows 2008) and are using it to protect a mix of system state and BMR (Windows 2008R2). The link between the primary and the off-site secondary DPM is a 2MB/s link. When we protect the BMR on the primary we see very little data change as expected the System Configuration has not altered significantly. However, when we create a recovery point on the secondary we see the entire data size get synced across to the secondary (in the data transferred column) which takes an enormous amount of time over a 2MB link and for 6 BMR servers tends to cause link congestion problems with other protection. What we don't see in the primary to secondary RP process is the RP volume increase by that amount of data that has been 'transferred' which would be correct if that amount of changed data were being sent across.

    For System State protection on Domain Controllers I am seeing a very large amount of change to the recovery point volume on every backup. I.e. The system state replica volume is 13GB and one RP will consume 7GB, two RP's will consume 14 GB, three RP's will consume 21GB and so on. In this case when performing System State on member servers I only see a very small amount of change for each RP in the 100's of MB's NOT GB's.

    Any ideas on why this could be occurring? We'd really like to use BMR but it's just not feasible with those Primary to Secondary change rates

    Sunday, August 15, 2010 11:52 PM

Answers

  • DPM uses inbox Windows Server backup for creating BMR backups. It has option to do incremental backups over wire. When you select a volume for BMR backup you can specify its performance settings to do performant backups vs less performant. When you performant backups it will create a shadow copy of the protected volume on the PS. Using this the next backup will be incremental and it only transfers the files changed between those two shadowcopies. This way the network bandwidth will be saved for you. On the contrary the source/BMR protected volume recieve a performance hit as there is always a shadow copy living on the system. You can launch wbadmin.msc on the protected computer and set its backup performance settings at volume level. In my knowledge, this option is available only for Windows Server 2008 R2 version. Hope this helps you resolve the issue.


    Thanks, Praveen D [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, February 10, 2011 6:54 PM

All replies

  • Hi Danny,

    Unfortunately we cannot see data size on primary protection (with System Protection / BMR) because it is directly written, not an agent-server job. However the same amount of data will be transferred which on the primary you only see as low RP consumption.

    What I think (but not sure) is happening is as follows;
    - rewrite of same data to same blocks causes volsnap to skip the copy_on_write for those blocks --> low RP volume consumption.
      Because you have no transferred data information this does not come to attention and local LAN speed can deal with it.
    - synchronizing agent-server (primary to secondary) shows as normal job with lot of transferred data but same happens --> low consumption.

    On DC's a relatively large part of what belongs to critical volumes changes, in your case summing to 7GB.
    Are you referring to transferred data or true consumption on the recovery point volume?

     Do you sync BMR daily ?


    \R2 This posting is provided "AS IS" with no warranties, and confers no rights
    Sunday, November 21, 2010 3:36 PM
  • The issue is the transfer of data - whether it consumes space on the RP volume or not. The time and bandwith taken to transfer such significant amounts of data cripples the link during that time. E.g. a 2mb/s link will transfer approximately 20GB of data in a 24 hour period. If 4 BMR's are taking place (in my case transferring across the link 28GB of data) - plus other regular File, Exchange and SQL data, I am going to miss recovery points on the day that BMR takes place even if I only do them once per week.

    ..... and yes, for domain controllers we do want to use BMR daily.

    Sunday, November 21, 2010 11:47 PM
  • Yes, I understood the link transfer is the issue.
    I'm afraid 2MB/sec is too little for this protection.


    \R2 This posting is provided "AS IS" with no warranties, and confers no rights
    Monday, November 22, 2010 7:56 AM
  • DPM uses inbox Windows Server backup for creating BMR backups. It has option to do incremental backups over wire. When you select a volume for BMR backup you can specify its performance settings to do performant backups vs less performant. When you performant backups it will create a shadow copy of the protected volume on the PS. Using this the next backup will be incremental and it only transfers the files changed between those two shadowcopies. This way the network bandwidth will be saved for you. On the contrary the source/BMR protected volume recieve a performance hit as there is always a shadow copy living on the system. You can launch wbadmin.msc on the protected computer and set its backup performance settings at volume level. In my knowledge, this option is available only for Windows Server 2008 R2 version. Hope this helps you resolve the issue.


    Thanks, Praveen D [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, February 10, 2011 6:54 PM
  • Praveen D -- I want to make sure I undertand you correctly.  Are you suggesting that on each server using BMR backup, we should change the setting in wbadmin.msc to do incremental backups, instead of full backups?  Or are you recommending that the setting be changed on the primary DPM server to affect the amount of data transferred when replicating data between the primary and secondary servers?

    Is DPM designed to work with these incremental backups, and could DPM still do a seamless recovery of the entire server from any date protected, or does it become more complicated due to the incremental backups?

     

    I thought DPM already used VSS to store only the changed data between backups, as opposed to storing a full backup each day?  And if the DPM server is in fact storing only the delta, why would the secondary DPM server be copying a full backup's worth of data over the network when it's backing up protected servers from the primary DPM server?  

     

    I had understood that it should work like this:

    DPM1 uses Windows Backup on FILESERVER1 to create an image of the drive.  The full backup of FILESERVER1 is stored on DPM1 as it's being created.  

    DPM1 compares the previous image backup of FILSERVER1 to the new backup, and saves only the data which has changed between the two backups.

    DPM2 is protecting DPM1, and the protected server data on it.  Since DPM1 still has the same original, unchanged, backup which DPM2 has already copied previously, DPM2 would now copy only the newly saved delta. This would ordinarily result in a relatively small amount of data transfer.

     

    It looks like instead DPM2 is pulling over the combined full & delta as a complete image, and again doing the same work DPM1 already did to identify and protect changed data. Is that correct?

    • Proposed as answer by NeighborGeek Thursday, December 29, 2011 2:25 PM
    Monday, April 4, 2011 4:39 PM
  • For anyone who happens to come across this thread looking for an answer, I did get an answer in another thread on a mostly unrelated topic. This was posted by Mike Jacquet:

    "As far as BMR is concerned, the reason why the secondary DPM server seems to replicate the whole BMR image each time is due to the way BMR is performed on the Primary. Windows Server backup on the protected server "overwrites" the entire BMR backup image .vhd file(s) on the DPM server, so the DPM Filter tracks those overwrites and the secondary needs to bring them over and apply the writes to it's copy, basically overwriting the entire image vhd file(s).   So we are only bringing over changed blocks, but every block has changed due to the overwrite method of BMR backups."

     

    Effectively, this is considered working as intented, and not a bug in DPM.  As such, it has not been addressed at all in DPM 2012 and will continue to transfer the full BMR backup to the secondary dpm server every time.


    • Edited by NeighborGeek Thursday, December 29, 2011 2:26 PM
    • Proposed as answer by NeighborGeek Tuesday, January 15, 2013 3:57 PM
    Thursday, December 29, 2011 2:25 PM
  • I need to deal with this issue almost daily.  Its a real pain and there is no way around it. It is functioning as per the product design.

    We have over 20 sites that connect to our WAN at 2, 10, 30 and 100Mbps. 

    Each site has 6VM's and a host server we need to protect.

    So that means we have no choice but to haul ~1.2TB of System State data accross our WAN, EVERY 24 HOURS, to our secondary DPM.

    Some of our links wont even sync System States in time.

    Just ridiculous.

    Microsoft - please consider allowing a synch frequency of greater than 24 hours (at the very least for system states).

    LET US DECIDE how often WE want to sync.

    Friday, February 10, 2012 5:30 AM
  • Hi Digijim,

    There is a workaround, although not ideal that might help you out if you want to change the Sync interval on the secondary. If you protect the replica volume path of the System State or BMR backup on the primary server (C:\program files\Microsoft DPM\DPM\replicas\etc.....) you can treat this as data that is protected as primary data and you can set the schedule.

    The drawbacks are that you will not be able to switch protection if the primary fails but it sounds like your main issue is getting the data off-site period at this point.

    Friday, February 10, 2012 6:21 AM
  • Have you found a solution to your problem?  I'm working with a similiar environment and I'd be really interested to hear your thoughts.
    Tuesday, February 21, 2012 10:27 AM