none
DPM 2010 Tape Expiration Overlap RRS feed

  • Question

  • I had an email protection group set to expire this weekend.  The tape management report said to put in a certain tape as it was going to expire.  So I did.  I put in another tape that was for the database group and a blank for the file group that I'd missed previously.  The database group worked fine, but the email group used the blank tape and not the tape set to expire.  I believe the problem was the date that it was going to expire was the day after it was set to run the job.  I've run into this before and I'm not sure how to combat it.  The week's end a day after they did last year, so something set to expire 365 days from last year will not be on a Saturday, but a Sunday meaning my weekly job running on a Saturday won't work.

    How do you combat this?  I already know that I can't force a tape to expire unless I stop the protection group.  More and more this seems less automated and more micromanagement and not at all helpful.

    Thursday, October 27, 2011 9:08 PM

Answers

  • FYI - we're investigating a fix in DPM for this issue - the problem is that when DPM writes a recovery point to tape, it sets the expiry date-time to the current date-time + retention period. 

    Since current date-time is ~ 1 minute ahead of the scheduled start date-time, the tape does not expire that RP until AFTER the next weeks scheduled backup time, thus we can't find a free tape.

    One of three things can occur:

    1) We fail the backup because a free tape cannot be found.
    2) We append to the tape instead of overwriting it and risk filling it up and having to ask for a new tape.
    3) We succesfully overwrite the tape and the backup suceeds.

     


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, December 3, 2011 5:44 PM
    Moderator
  • Hello,

    We do have a fix for this issue and it is currently being tested.  If you would like the private to test yourself, please open a support case and we can provide the test fix.   FYI, assuming it works as designed, it will eventually be includd in the next DPM 2010 rollup and eventually also added to DPM 2012 Sp1 when released.

    FIXED - you need to add a registry value to enable it.

    •Tapes are not reusable until the day after the day of expiry. This is true because DPM waits until midnight to run the reclamation job that marks tapes as reusable. If a user wants a tape to be reusable on the same day on which it expires, the user should follow these steps:

    1.Create the following registry key:


    HKLM\Software\Microsoft\Microsoft Data Protection Manager\Configuration\MediaManager

    2.Create the ExpireDatasetOnDayStart 32-bit DWORD value, and then set its value data to 1.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.


    Wednesday, February 8, 2012 3:04 PM
    Moderator

All replies

  • Hi,

    Yes, that is the way expiration works, there is date&time associated with each recovery point, and until the last recover point expires, the tape cannot be used again for rotation.  I don't have a tape management report in front of me, but I believe it does list a date and time for which a tape should be due back, and I guess best practice would be to return it after it's overdue.

     

    There are two workarounds:

    1) You can always free a tape uding this power shell script: http://blogs.technet.com/dpm/archive/2007/09/04/cli-script-force-mark-tape-as-free.aspx

    2) You can use this solution to expire tapes 24 hours earlier that when DPM would expire them.

    http://social.technet.microsoft.com/Forums/en-US/dpmtapebackuprecovery/thread/62717f9d-1a92-47ca-a461-76f740ffed90/

     <snip>

    /*
    2010Sep17 Oliver Wilcock: script for DPM2010 to expire tapes 24 hours earlier than DPM would expire them.
    -- This is useful for weekly rotations and other common sense interpretations of retention periods.
    -- Tapes must be online to be affected by the script.
    -- After this script the UI will not show the affected tapes as expired, but the tape engine appears to recognize them as expired and will use them.
    -- To make the UI show the tapes as expired you must restart the UI or trigger a fast inventory. A FastInventory can by triggered with PowerShell like this:
    -- get-dpmlibrary -DPMServerName backup1 | where {!$_.IsOffline} | Start-DPMLibraryInventory -fastInventory
    
    Disclaimer: This script seems to work for me, but I'm making no warranties or claimed fitness to your purpose.
    Known problem: The choice of 0600 for the expiry dates is to accomodate UTC -0500 timezones with DST and my imperfect understanding of how to manipulate dates using Transact SQL.
    Install:
    0. Backup your database. You'd better know how to restore it, because, you know, DPM doesn't know how to do that. Doh!
    1. Create an SQL Agent task and schedule it for a suitable time before your tape backups begin and when your media will be loaded. I use 18:10.
    2. Set the owner of the task appropriately. I used the local account created by the installer MICROSOFT$DPM$Acct.
    3. Create a step, set the type to Transact-SQL, set the database to the DPM database (ie DPMDB), paste this code as the only step.
    4. Test it.
    5. Curse proprietary, closed source, over priced, feature poor, time wasting, software but be thankful for the rare times that it can be hacked, cobbled together and futzed-with-into-the-wee-hours-for-weeks-on-end into a small fraction of a make-shift solution.
    6. Whew!
    
    */
    begin transaction
    update 
     tbl_MM_PhysicalDataset
    
     -- the the ExpiryDate to today at 0600
     set ExpiryDate = DATEADD(HOUR,6, convert( datetime,convert( date, GETDATE() ) ) )
     , LifeStatus = 2
     WHERE
     Datasetid in
     (
    select Dataset.Datasetid
     -- , barcodevalue
     FROM  dbo.tbl_MM_Media AS Media INNER JOIN
     dbo.tbl_MM_ArchiveMedia AS ArchiveMedia ON ArchiveMedia.MediaId = Media.MediaId INNER JOIN
     dbo.tbl_MM_MediaMap AS MediaMap ON MediaMap.MediaId = ArchiveMedia.MediaId INNER JOIN
     dbo.tbl_MM_PhysicalDataset AS Dataset ON Dataset.DatasetId = MediaMap.DatasetId
     WHERE
     Location = 1 -- Media is online
     and ExpiryDate between -- Expiring in the next 24 hours. UTC or local timezone?
      convert(date, GETDATE() )and DATEADD(HOUR,24,GETDATE() )
      -- order by barcodeValue
     )
    go
    
    update 
     tbl_MM_ArchiveMedia
    
     set IsOmidChangeNeeded = 1, DatasetsState = 3
     WHERE
     MediaId in
     (
     select ArchiveMedia.MediaId
     FROM  dbo.tbl_MM_Media AS Media INNER JOIN
     dbo.tbl_MM_ArchiveMedia AS ArchiveMedia ON ArchiveMedia.MediaId = Media.MediaId INNER JOIN
     dbo.tbl_MM_MediaMap AS MediaMap ON MediaMap.MediaId = ArchiveMedia.MediaId INNER JOIN
     dbo.tbl_MM_PhysicalDataset AS Dataset ON Dataset.DatasetId = MediaMap.DatasetId
     WHERE
      DatasetsState = 2 -- not expired/reclaimed
      and Location = 1 -- Media is online
      and ExpiryDate between -- Expiring in the next 24 hours. UTC or local timezone?
      convert(date, GETDATE() ) and DATEADD(HOUR,24,GETDATE() )
     )
    
    go
    commit
    
    
    >snip<
    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, October 28, 2011 4:27 AM
    Moderator
  • I'll look at this, but it upsets me that I need a work around.  There are default short and long term protection cycles, weekly and monthly.  It's my understanding that generally you keep backups for a year.  This isn't something I just made up, it's SOP for most places running backups.  Even with a previous backup system we kept data for at least a year, but a year minimum.

    I think it's bad programming, knowing that unless it's a leap year, your year isn't going to expire until the day after the protection job is set to run.  It doesn't appear anyone thought this through.  And now we have custom scripts and commandlets to work around it?

    At this point, you might as well force expire your tapes immediately after they're used, or spend the upcoming week doing the same.  The automation on this is totally lost.

    Never had this problem with Backup Exec AND they handle non HyperV virtual shares.
    • Edited by brollman Friday, October 28, 2011 2:10 PM
    Friday, October 28, 2011 2:09 PM
  • Is this script a one time thing or do you have to run it against the DB before each tape job?  I don't know very much SQL so  I can't tell from the script and step 1 makes me think it has to be done each time.
    Friday, October 28, 2011 8:03 PM
  • Hi,

    The SQL job you create in 1) should be a reoccuring job so it runs at the same time every day, so schedule it to run after you plan on replacing media in the tape library, but before the next round of scheduled backups start.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, October 29, 2011 5:51 PM
    Moderator
  • FYI - we're investigating a fix in DPM for this issue - the problem is that when DPM writes a recovery point to tape, it sets the expiry date-time to the current date-time + retention period. 

    Since current date-time is ~ 1 minute ahead of the scheduled start date-time, the tape does not expire that RP until AFTER the next weeks scheduled backup time, thus we can't find a free tape.

    One of three things can occur:

    1) We fail the backup because a free tape cannot be found.
    2) We append to the tape instead of overwriting it and risk filling it up and having to ask for a new tape.
    3) We succesfully overwrite the tape and the backup suceeds.

     


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, December 3, 2011 5:44 PM
    Moderator
  • Any ideas of when this fix would be included in a hotfix?  I already knew there was an issue, knowing MSFT knows doesn't fix it.

    Friday, January 6, 2012 7:48 PM
  • Would be nice if you can fix this issue. Same problem here as mentioned above.

    Thanks,
    Peter

    Wednesday, February 8, 2012 8:13 AM
  • Hello,

    We do have a fix for this issue and it is currently being tested.  If you would like the private to test yourself, please open a support case and we can provide the test fix.   FYI, assuming it works as designed, it will eventually be includd in the next DPM 2010 rollup and eventually also added to DPM 2012 Sp1 when released.

    FIXED - you need to add a registry value to enable it.

    •Tapes are not reusable until the day after the day of expiry. This is true because DPM waits until midnight to run the reclamation job that marks tapes as reusable. If a user wants a tape to be reusable on the same day on which it expires, the user should follow these steps:

    1.Create the following registry key:


    HKLM\Software\Microsoft\Microsoft Data Protection Manager\Configuration\MediaManager

    2.Create the ExpireDatasetOnDayStart 32-bit DWORD value, and then set its value data to 1.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.


    Wednesday, February 8, 2012 3:04 PM
    Moderator