none
Making the BMR backup efficient RRS feed

  • Question

  • Separated into multiple posts:

    The BMR backup is really not feasible for bandwidth limited remote servers as it is implemented. As noted earlier in the forum, the system state and BMR backups send all the data to the DPM server every time. The following setup will allow significant efficiency gains by using RDC as implemented in DFS. It involves setting up DFS replication and modifying the bmrbackup.cmd script to use that share. I have not done anything with the lhsystemstate.cmd command since I'm not using it. The instructions are a bit crude but should at least get you started in the right direction. Caveats:

    1.    My servers are connected by IPSec tunnels and have a full two-way trust in place.

    2.    I'm only using 2008 R2 so there may be some differences in the setups if you are using something else.

    3.    The process does not care about disk usage on the remote machine or the DPM server.  The BMR backups may double the demand for disk space because of the staging leveraged by DFS

    4.    I’ve created a simple script to wait on the replication after the backup.  This helps DPM realize a successful backup and create necessary XML files correctly on the DPM server.  The script needs to be copied into the bin directory of the DPM agent install folder.

    5.    I have not tested this setup in a robust fashion.  It is not supported and I in no way take any responsibility for how it is used or the results of it.  You’re responsible for testing it in your environment and making sure your BMR is successful.

    6.    My results show that the amount of data transferred is very small with each incremental BMR.  Your results may vary.

    7.    I’m running Windows 2008 R2 Data Center edition on my DPM server.  This gives DFS the ability to use cross-file RDC.  I’m not sure that it will make a difference in your results but it in my caveats!

    For Microsoft, it seems like a simple way to make BMR better would be to provide a command-line utility that does RDC synchronization using the API for RDC. This could be deployed with the agent and used to sync the local backup for both BMR and SystemState to the remote DPM share within the bmrbackup.cmd script. It is a quick, relatively easy implementation that still leverages the wbadmin utility, but it does require the BMR to be housed on the local machine. I'm still looking for such a utility and if I find it, will incorporate it into the bmrbackup.cmd script rather than using DFS and waiting for the sync. It seems like this would be a good option to consider if a true block-level BMR backup is not going to be developed outside of wbadmin.


    Rob
    • Moved by Larry Yin Wednesday, March 28, 2012 10:25 PM (From:DPM 2012 Beta - Locked)
    Wednesday, January 25, 2012 7:50 PM

All replies

  • 1. Set up BMR protection in DPM for the server as you would normally. You can set up the replica manually or send it over the wire. You should wait to copy over the replica until later if you are creating it manually.

    2. Create a folder that will always be excluded from any volume or share level protection and will house your BMR backup. Otherwise you will be sending the same data over and over through volume or share protection. I use a standard folder that I put copies of media and other things that I don't need to protect via backup. To make it easy in the instructions, assume the share and foldername are BMR.

    3. Share the folder just created with at least SYSTEM access to the share and NTFS permissions. I use SYSTEM, Local Administrators and Domain Admins.

    4. Find out the path to the BMR protection on the DPM server. The path can be obtained by going to Protection and selecting the PG with the BMR setup you created in 1. Click to view details. There will be a long path to the location of the folder in DPM. You will need to know this in later steps.

    5. Change the bmrbackup.cmd script to the following:

    @echo off
    
     
    
    if not exist %SystemRoot%\system32\wbadmin.exe goto returnError
    
     
    
    rem Start a wbadmin.exe to create a BMR backup
    
    start  /WAIT %SystemRoot%\system32\wbadmin.exe start backup -allcritical -quiet -backuptarget:\\%COMPUTERNAME%\BMR
    
     
    
    rem Exit if there is a backup error
    
    if NOT %ERRORLEVEL% == 0 goto returnError
    
     
    
    rem Get the new folder for the BMR
    
    for /f "delims=|" %%a in ('dir /b \\localhost\BMR\WindowsImageBackup\%COMPUTERNAME% ^| find "Backup"') do set newfldr=%%a
    
     
    
    rem Call vbscript program to wait on sync to complete and update event log
    
    %windir%\system32\cscript.exe verifysync.vbs "%newfldr%" "%1" "\\%COMPUTERNAME%\BMR"
    
     
    
    rem If return error code, Pass this error code to caller else check whether it actually succeeded or not
    
    if %ERRORLEVEL% == 0 goto returnSuccess
    
    exit /B %ERRORLEVEL%
    
     
    
    :returnSuccess
    
    exit /B 0
    
     
    
    :returnError
    
    exit /B 1

    6. Create verifysync.vbs in the bin directory of DPM. The contents should be as follows:

    ' Retrieve command line parameters
    bkuppath = wscript.arguments.item(0)
    dpmpath = wscript.arguments.item(1)
    hostpath = wscript.arguments.item(2)
    
    ' Standard objects
    Set WShell = WScript.CreateObject("WScript.Shell")
    set fso = CreateObject("Scripting.FileSystemObject")
    Set env = wshell.environment("Process")
    
    ' Set up the folder paths
    strComputer = env.Item("Computername")
    srcfldr = hostpath & "\WindowsImageBackup\" & strComputer & "\" & bkuppath
    destfldr = dpmpath & "\WindowsImageBackup\" & strComputer & "\" & bkuppath
     
    ' Log the two folders and start time 
    LogMsg = "Beginning monitor of source folder """ & srcfldr & """ and destination folder """ & destfldr & """."
    wshell.LogEvent 4, LogMsg
    
    ' Open the source folder
    set src = fso.GetFolder(srcfldr)
    
    ' Enter loop to check if file exists and that dates/times are the same at source and destination.  Loop every minute for a maximum of 3 hours.
    cnt = 0
    syncsuccess=1
    do while cnt < 180
       syncdone = 0
       for each f in src.Files
          destfile = destfldr & "\" & f.name
          if NOT fso.FileExists(destfile) then syncdone=syncdone+1
          if fso.FileExists(destfile) then
             set fdest = fso.GetFile(destfile)
             if f.Size <> fdest.Size then syncdone=syncdone+1
             if f.DateLastModified <> fdest.DateLastModified then syncdone=syncdone+1
             set fdest = nothing
          end if 
       next
       if syncdone=0 then
          syncsuccess=0
          exit do
       end if
       wscript.sleep 60000
       cnt = cnt + 1
    loop
    
    ' If the sync was successful before the timeout, then log success and return 0
    if syncsuccess=0 then
       LogMsg = "Sync succeeded before the timeout for " & src.Files.Count & " files."
       wshell.LogEvent 4, LogMsg
       wscript.quit 0
    end if
    
    ' If the sync was not successful, then log an error and return 1
    if syncsuccess<>0 then
       LogMsg = "Sync failed to complete before the timeout " & src.Files.Count & " files."
       wshell.LogEvent 1, LogMsg
       wscript.quit 1
    end if
    

    7. Manually run the backup to create the replica if you didn't do it automatically before. You will need to copy the data to the DPM server like you would for any other manual replica creation process.

    8. If not previously installed, install the DFS Replication service under Roles ->File Services -> Role Services -> Add role services. You do not have to install DFS Namespaces. Do this for both the DPM server and the protected server.

    9. Make sure the staging quote for the DFS replication is larger than your VHD file from the BMR backup. Otherwise you will not get an efficient transfer. The volume on the DPM server may need to be increased to accommodate this increased demand for staging. Or you can move the staging folder to a different volume on the DPM server.

    10. On the DPM server or the protected server, launch the DFS Management

    • New replication group
    • Expand Replication
    • Choose Multipurpose replication group
    • Name: <computer> BMR replacing <computer> with the FQDN of the protected server. The naming convention can be whatever you like but you should create unique DFSR setups for each BMR. Using the FQDN in the name helps keep these easy to understand.
    • Add in the protected server and the DPM server. This process assumes you have a trust in place so both can be easily added.
    • Full Mesh
    • Any special configuration options should be done here. In this scenario, the bmrbackup.cmd is modified to wait on the sync so if the sync isn’t allowed all the time, then the script may fail unnecessarily.
    • Primary member -> Protected server
    • Browse to the folder created in step 2.
    • Enabled -> Browse to the folder defined by the path identified in step 4.
    • Finish out the wizard
    • After creating the DFSR setup, go into the DFS Management and right-click on each member choosing Properties. Then go to the advanced tab and uncheck the "Move deleted files…" checkbox.

    Now anytime the BMR runs the data will use RDC to replicate the backup to the DPM server.

    DPM will still track the success of the backups and report to you. You will need to verify that replication is working. We are using SCOM to monitor DFS.

    The –allcritical switch also includes volumes that I don’t want to include (yes, my choice and risk). To address this, you can add the following vbscript to your DPM\bin directory and modify it to meet your needs. Then include a line in the BMR backup to run the script before the wbadmin command. This script excludes any service installed on the D: drive since that was what was causing the D: volume to be included in the allcritical backup. There may be other reasons your volume is getting included. Use the diskshadow output of the System Writer metadata to see why it is included if the script doesn't work.

    Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")

    Set objServices = objWMIService.ExecQuery("Select * from Win32_Service",,48)

    dim svclist()

    cnt = 0

    For Each svc in objServices

    if Instr(1,svc.Pathname,"d:\",1) > 0 Then

    redim preserve svclist(cnt)

    strPath = Replace(svc.Pathname,"""","")

    svclist(cnt)=strPath

    cnt = cnt + 1

    end if

    Next

    Const HKEY_LOCAL_MACHINE = &H80000002

    Set oReg = GetObject("winmgmts:\\.\root\default:StdRegProv")

    oReg.SetMultiStringValue HKEY_LOCAL_MACHINE,"SYSTEM\CurrentControlSet\Control\SystemWriter","ExcludedBinaryPaths",svclist


    Rob
    • Edited by ip-rob Wednesday, January 25, 2012 7:59 PM Cleanup formatting
    Wednesday, January 25, 2012 7:53 PM
  • I was having some troubles getting the scripts to run and realized that the default directory for the bmrbackup.cmd is c:\windows\system32.  So I put in this line:

    rem Get the path for DPM binaries
    for /f "tokens=1-2*" %%a in ('reg query "HKLM\Software\Microsoft\Microsoft Data Protection Manager\Setup" /v "InstallPath" ^| find "REG_SZ"') do set dpmpath=%%cbin

    to get the DPM path.  The environment variable %dpmpath% needs to be put in front of all the scripts and any output you want to log into the DPM\bin directory


    Rob
    Thursday, January 26, 2012 4:39 AM
  • I was having some troubles getting the scripts to run and realized that the default directory for the bmrbackup.cmd is c:\windows\system32.  So I put in this line:

    rem Get the path for DPM binaries
    for /f "tokens=1-2*" %%a in ('reg query "HKLM\Software\Microsoft\Microsoft Data Protection Manager\Setup" /v "InstallPath" ^| find "REG_SZ"') do set dpmpath=%%cbin

    to get the DPM path.  The environment variable %dpmpath% needs to be put in front of all the scripts and any output you want to log into the DPM\bin directory


    Rob

    Could you please elaborate on this? I'm trying out the script and can't get it to run. Where should I add that line?

    Best Regards

    Mikael Andersson

    Wednesday, February 22, 2012 1:05 PM
  • I put that line in the beginning of the bmrbackup.cmd script right after the line

        if not exist %SystemRoot%\system32\wbadmin.exe goto returnError

    You could also put the path in as a static value.  Be careful about quotes.  The %dpmpath% environment variable will have spaces in it so you will need to quote anywhere it is used (e.g. %windir%\system32\cscript.exe "%dpmpath%\verifysync.vbs" "%newfldr%" "%1" "\\%COMPUTERNAME%\BMR$").

    You may want to run the bmrbackup.cmd manually within the DPM bin directory and see if things are working as expected.  I commented out the verifysync.vbs script for testing. 

    This process became a little cumbersome to scale out without a nice replication command line utility. 


    Rob

    Wednesday, February 22, 2012 1:34 PM
  • Thank you!
    That's where I put it. 
    About the verifysync.vbs script, should that be placed in the agent folder on the protected server?
    And the bmrbackup.cmd I should edit is also on the protected server?

    Best Regards

    Mikael Andersson


    • Edited by Stuvning Wednesday, February 22, 2012 1:48 PM
    Wednesday, February 22, 2012 1:46 PM
  • Yes, it should be put in the bin folder on the protected server.  All script changes should be in the bin folder.

    Rob

    Wednesday, February 22, 2012 1:49 PM
  • OK, thanks!

    Going to be fiddling around some with it. In the meantime, could you explain how it is more efficient than
    the DPM implementation? I don't think I understand, doesnt it do the same thing DPM does just utilizing DFS?
    Copying the whole backup due to the way BMR is designed around VHD-files?

    Best Regards,
    Mikael Andersson


    • Edited by Stuvning Thursday, February 23, 2012 1:14 PM
    Thursday, February 23, 2012 8:42 AM
  • The regular BMR backup sends the entire vhd file across the network.  The DFS-R setup sends only the block-level changes.  The actual BMR backup is written locally and then DFS-R replicates it to the DPM machine.  In my testing, a 20GB vhd had only 200MB sent over the network.  It took as long as 3 hours to replicate...because DFS-R was calculating the blocks that were different. 


    Rob

    Thursday, February 23, 2012 2:03 PM
  • Ran into another issue. Getting this error in event viewer on my DPM-server:

    Additional Information: 
    Error: 1005 (The volume does not contain a recognized file system.
    Please make sure that all required file system drivers are loaded and that the volume is not corrupted.) 
    Volume: A904DBFF-5E2E-11E1-804F-001B245EC1A3



    Best Regards,

    Mikael Andersson



    • Edited by Stuvning Friday, February 24, 2012 12:08 PM
    Friday, February 24, 2012 12:07 PM
  • I think that is an unrelated error.  I've never received an error like that when testing the DFS-R method.  You may want to post that as a separate thread and see if someone can help troubleshoot the issue.

    Rob

    Friday, February 24, 2012 1:53 PM