none
DPM 2010 Partition Limit, Maximum 600 partitions RRS feed

  • Question

  • Hi,

    As you are aware DPM backs up to partitions its creates on an unformatted, however this has left our environment with a problem as there appears to be a limit of 600 partitions per drive within dpm and we have reached this limit.

    So my question is can this be bypassed maybe via a regedit? and is this a DPM limit or is it a windows limit that DPM is reporting.

     

    Thanks,

    Allan

    Monday, September 6, 2010 9:26 AM

Answers

  • Hi

     

    As Rod mentioned - the LDM database does have has limitations, so our guidelines for the 300 data sources is based on those limitations.  However, Windows does support more than 600 volumes, in fact, it can actually host greater than 2500 simpe dynamic volumes.  The problem comes in when DPM starts extending the replica and recovery point volumes due to poor pre-planning of space requirenents for the various data sources under protection.  If DPM extends them too many times, you bump into a hard limitation where you cannot add new volumes or extend exisiting volumes.

    To overcome that limitation once it's reached, you can identify which data source volumes have the most extents, then migrate the data source to DPM Custom volumes on BASIC DISKS using the MigrateDatasourceDataFromDPM.ps1 power shell command.

    Using MigrateDatasourceDataFromDPM: http://technet.microsoft.com/en-us/library/dd282970.aspx

    How to Use The Migratedatasourcedatafromdpm.Ps1 DPM Powershell Script to Move Data: http://blogs.technet.com/b/askcore/archive/2009/06/22/how-to-use-the-migratedatasourcedatafromdpm-ps1-dpm-powershell-script-to-move-data.aspx

    What volume to migrate first?: http://blogs.technet.com/b/dpm/archive/2010/03/30/what-volume-to-migrate-first.aspx

    Space needed to migrate a disk: http://blogs.technet.com/b/dpm/archive/2010/06/05/space-needed-to-migrate-a-disk.aspx

    Once you migrate it, wait for the recovery points to expire and those dynamic volumes will be deleted by DPM and free up some records in the LDM database.  Do that for the data sources containing the most extents so to free up the most LDM records.  Then once you see that those volumes have been deleted (after the recovery points expire) , add one more dynamic disk, and migrate the data sources on the custom volumes back to the new dynamic disk that has all free space, that will ensure only two records are used, one for the replica and one for the recovery point volume.   Then you can start migrating other datasources with lots of extents to the new dynamic disk and free up even more records.  Over time, you can manage to get most of your volumes back into contiguous space by migrating data sources between disks that have the most contiguous free space and reduce your LDM record count even further.  The important thing to remember is you can only migrate a data source one time until the old volumes get deleted after the recovery points expire, so depending on the retention range you have this could take days, weeks or months.

    I guess the take away is to do your storage planning up front so DPM does not need to keep growing volumes, because each time it does, it uses more LDM records which have a finite number.


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, September 6, 2010 4:36 PM
    Moderator
  • Hi,

     

    Looking at the numbers you supplied me, you are in great shape, you are not at risk of running out of LDM database space so I would not worry about having to migrate.  The total extents are including the ones used by volumes (each volume is created in one extent), so 645-582=63 real extentions.


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, September 7, 2010 2:56 PM
    Moderator

All replies

  • It is an operating system limitation and cannot be overridden.

    The Logical Disk Manager Database (used with Dynamic Disks like the ones in the DPM storage pool) has a limit to the number of slots/records it contains.  This is why DPM says not to protect more than 300 datasources with a single DPM server (which results in 600 volumes, one for the replica and one for the recovery point). 

    Are you protecting 300 data sources?  You are definitely overdue for adding another DPM server to your infrastructure.

    • Proposed as answer by Rod Savard Monday, September 6, 2010 2:53 PM
    Monday, September 6, 2010 2:50 PM
  • Hi

     

    As Rod mentioned - the LDM database does have has limitations, so our guidelines for the 300 data sources is based on those limitations.  However, Windows does support more than 600 volumes, in fact, it can actually host greater than 2500 simpe dynamic volumes.  The problem comes in when DPM starts extending the replica and recovery point volumes due to poor pre-planning of space requirenents for the various data sources under protection.  If DPM extends them too many times, you bump into a hard limitation where you cannot add new volumes or extend exisiting volumes.

    To overcome that limitation once it's reached, you can identify which data source volumes have the most extents, then migrate the data source to DPM Custom volumes on BASIC DISKS using the MigrateDatasourceDataFromDPM.ps1 power shell command.

    Using MigrateDatasourceDataFromDPM: http://technet.microsoft.com/en-us/library/dd282970.aspx

    How to Use The Migratedatasourcedatafromdpm.Ps1 DPM Powershell Script to Move Data: http://blogs.technet.com/b/askcore/archive/2009/06/22/how-to-use-the-migratedatasourcedatafromdpm-ps1-dpm-powershell-script-to-move-data.aspx

    What volume to migrate first?: http://blogs.technet.com/b/dpm/archive/2010/03/30/what-volume-to-migrate-first.aspx

    Space needed to migrate a disk: http://blogs.technet.com/b/dpm/archive/2010/06/05/space-needed-to-migrate-a-disk.aspx

    Once you migrate it, wait for the recovery points to expire and those dynamic volumes will be deleted by DPM and free up some records in the LDM database.  Do that for the data sources containing the most extents so to free up the most LDM records.  Then once you see that those volumes have been deleted (after the recovery points expire) , add one more dynamic disk, and migrate the data sources on the custom volumes back to the new dynamic disk that has all free space, that will ensure only two records are used, one for the replica and one for the recovery point volume.   Then you can start migrating other datasources with lots of extents to the new dynamic disk and free up even more records.  Over time, you can manage to get most of your volumes back into contiguous space by migrating data sources between disks that have the most contiguous free space and reduce your LDM record count even further.  The important thing to remember is you can only migrate a data source one time until the old volumes get deleted after the recovery points expire, so depending on the retention range you have this could take days, weeks or months.

    I guess the take away is to do your storage planning up front so DPM does not need to keep growing volumes, because each time it does, it uses more LDM records which have a finite number.


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, September 6, 2010 4:36 PM
    Moderator
  • Hi,

    Thanks for your input we have about 5Tb of unused disk space in the RAID volume being used for DPM backup if i recofigure the RAID into two drives would this work?

    Yes we are backing up alot this is due to the fact that we are backing up each database seperatly, which we do not want to change.  We only have a few more servers to add and we would have plenty of space for expantion.

    Mike i dont quite follow the migrating you suggest would this be storing the backup to another drive or would it consolidate extention partitions into a single partition freeing up partitions alocation? Is there any chance you could explain this a bit more basically for me?

    He is the the usage of the disk/partitions.

    Total volumes        : 582
    Total extents        : 645
    Total data sources   : 285
    Number of non-colocated data sources that can still be added: 9

    Having read what you've written 645 seems like alot of extends but they must not be counted as partitions. Are you saying if i migrate and get rid of the extends i'll be able to fit more partitions?

     

    Thanks,

    Allan

    Tuesday, September 7, 2010 9:38 AM
  • Hi,

    I think i'm going to go down the route of consolidating the databases we back-up into groups as this should solve the problem, i was hoping to avoid it but there you go.

    Most of what i'm backing up is databases so i should be able to get the number of partitions down, just need to wait for them to expire.

    Thanks again for your help.

    Allan

    Tuesday, September 7, 2010 12:56 PM
  • Hi,

     

    Looking at the numbers you supplied me, you are in great shape, you are not at risk of running out of LDM database space so I would not worry about having to migrate.  The total extents are including the ones used by volumes (each volume is created in one extent), so 645-582=63 real extentions.


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, September 7, 2010 2:56 PM
    Moderator