none
After Experienced opinions on this ... RRS feed

  • Question

  • Hello,

    We are planning on switching to DPM for all our windows operations servers, Sharepoint, AD, SQL, that kind of thing.

    However we also have some Windows servers with large volumes of scientific data, total of 45TB the largest single disk being 30TB and growing ~1TB/Month. The data once created and processed is rarely written too again, but does need to be retained on disk for read access.

    I have 70TB disk and a 45 slot LTO4 tape library which can be dedicated to just backing up the scientific data, all the other windows servers will be on a different DMP server/storage/tape.  More Disk can be added later if needed. I'd like to know peoples opinion as to whether DPM will cope with this?

     

    thanks

     

     

     

     

     

     

     


    • Edited by Nigel_CR Thursday, December 22, 2011 10:33 AM
    Thursday, December 22, 2011 10:29 AM

All replies

  • Hi,

    Yes, DPM can protect that volume with a few considerations based on http://technet.microsoft.com/en-us/library/ff399280.aspx

    1) Each disk in the DPM storage pool need to be 17TB or smaller and if over 2TB must be converted to GPT before adding them to the storage pool.
    2) DPM Supports a maximum of 80TB in the DPM Storage pool.
    3) If the volume consists of more than ~3 million files, tape based backup may fail with an error:

          The operation failed because of a protection agent failure. (ID 998 Details: The parameter is incorrect (0x80070057))

    This is fixed in DPM 2012 to be released next year - but we have a private fix for DPM 2010 V3.0.7707 if you cannot wait.
     


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, December 22, 2011 6:46 PM
    Moderator
  • Thanks Mike,

    I also read that the restore point maximum size is 40TB?

    So, on paper, it should be OK, but what's it like in practice? What's the risk of it becomming a backlogged quagmire when running maintenance tasks on the storage pool for example? or should DPM be able to cope with this easily?

     

     


    • Edited by Nigel_CR Wednesday, January 4, 2012 1:23 PM
    Wednesday, January 4, 2012 1:22 PM
  • The problem you are going to face is the length of time that a consistency check will run if the volume ever gets into an inconsistent state.  The Consistency check has to check every file on the volume and based on disk / network performance and the number of file on the volume, this could take days to complete.  You also need to think about full restore time on a volume that large with that many files. 

    Ideally, a CC would take no longer than 24 hours for any protected volume in a worst case senario.  I'm not sure if you could re-arrange the data to mountpoints.   Basically have a small ntfs volume to be the host volume for mountpoints, then have multiple larger volumes be mounted under directories that host the data and are shared so clients can get at the data.        

    In my example below,  Host_Volume (U:) is just a very small ntfs volume that only holds the mount point directories and shares. 

    UserShare# is a folder that is shared for user access and is the empty mountpoint folder.   Mounted_Volume# is the underlying volume that holds the directories and data.  DPM protects U: and all of the Mounted_VolumeX disks.   That way a chkdsk and / or a DPM CC only needs to run against the subset of data files on one volume and won’t take as long.

    Host_Volume (U:)
       UserShare1 -->Mounted_Volume1
                              DataDir1
                              DataDir2
                              DataDir3
       UserShare2 -->Mounted_Volume2
                              DataDir4
                              DataDir5
                              DataDir6
       UserShare3-->Mounted_volume3
                              DataDir7
                              DataDir8
                              DataDir9


    You can “grow” the Mounted_volume at the SAN level if you need more space on that disk for users data, then use diskpart.exe command to grow the NTFS file system into the new free space, that way you can start with smaller LUN’s and grow them over time as new users / data gets added.

    File Systems
    http://technet.microsoft.com/en-us/library/cc938934.aspx

     


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, January 4, 2012 8:11 PM
    Moderator
  • We did look into using mount points, but the data is managed by the application, there is no ability to break it up into subfolders in a way that we can control outside of the application, so we're stuck with the existing structure where a subfolder is created for each batch process under a single folder.

    Restore time isn't really critical, so long as we can restore eventually.

    Missing out on backing up a few days a month would be acceptable too, the experiments can be re-done should we lose any data during that time window, but what sort of frequency should I typically expect to have to run consistancy checks? once a week would be a problem, but every few months we might be able to live with.

     

     

     

     

     

     

    Thursday, January 5, 2012 4:38 PM
  • Hi,

    Typically a consistency would only need to be done under one of a few senarios:

    1) A scheduled synchronization fails in the middle of a transfer so we only brought over some changes and applied them to the replica.  DPM would mark the replica inconsistent and a CC would need to be ran.  If you have an unreliable or slow link between the DPM server and the protected server, then that might increase the likely hood of this occuring.

    2) An unexpected system crash on the protected server and the DPM filter detected it lost track of a change on the protected volume.

    3) An NTFS USN Journal wrap occured on the protected servers protected volume.  Very rare especially if the volume is not heavily modified like in your case, and USN journal can be increased if necessary should that ever occur. 

    I would say to go ahead and give DPM a whirl and see how it does in your environment.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, January 5, 2012 5:21 PM
    Moderator
  • that sounds OK, yes, I think we'll give it a try.

    thanks for your help,

    Friday, January 6, 2012 10:46 AM