none
Weekly Tape Rotation in DPM 2010 RRS feed

  • Question

  • We perform daily backups to tape of a configured protection group. The retention period is 1 week. I believe this may be causing an issue when the following week comes around - the backup appears to fail due to the fact the the previous weeks data has not yet expired.

    I want to configure the job so that every Monday the data on the tape will be appended to, where possible, or overwritten - can I set a retention range of 6 days for a daily backup so that when the week comes arounf the data has expired allowinf the tape to be written to.

    Thanks in advance.

    Monday, July 19, 2010 2:54 PM

Answers

  • We are actually using 5 tapes per week Monday to Friday so in the scenario above where we have overwritten the previous weeks Monday tape we can use the other daily tapes to get the data we are looking for.

    I dont belive this is an unusual tape rotation policy and would be very common especially with small businesses. They would use these daily tapes and then 12 monthly tapes for the end of month backups.

    This does not appear to be an issue anymore since I enabled co-location on the library - not sure why this was?

    I will mark this question as answered although my intial query is still outstanding.

     

    Monday, August 16, 2010 11:11 AM

All replies

  • Is there nobody else having the same problem?  Or is the lack or response because die-hard DPM users have given up on a weekly rotation?

    Is this a new frustration in DPM2010?  Was is possible in DPM2007 to set the long-term retention range to days?  I tried setting it to 0.9 weeks but the evil wizard changed it to one week.  Grr.

    Can we get Microsoft to change the definition of "one week" to 7 days minus one hour?  That would be sensible.

    Think about this one: Why does the evil wizard show me a retention in Weeks for Weekly backups, Years for Yearly backups, Months for Monthly backups, but Weeks for Daily backups?  Oh, right! Because the wizard is evil!

     

    Friday, August 13, 2010 2:44 PM
  • You can never achieve the goal of using a single tape for weekly backups, nor should you want to.  Think about the senario that on Monday we overwrite last weeks tape, but you needed an earlier version of a file from that backup tape - oops.

     

    Lets be practical, you need a minumum of two tapes for rotational purposes and to be honest, you should keep one offsite at all times, so that requires three tapes.  Tapes are not that expensive and will actually last longer if you rotate them.   

    You should be utilizing the tape management report to tell you when a tape should be back onsite to be reused.


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, August 13, 2010 4:23 PM
    Moderator
  • We are actually using 5 tapes per week Monday to Friday so in the scenario above where we have overwritten the previous weeks Monday tape we can use the other daily tapes to get the data we are looking for.

    I dont belive this is an unusual tape rotation policy and would be very common especially with small businesses. They would use these daily tapes and then 12 monthly tapes for the end of month backups.

    This does not appear to be an issue anymore since I enabled co-location on the library - not sure why this was?

    I will mark this question as answered although my intial query is still outstanding.

     

    Monday, August 16, 2010 11:11 AM
  • HI All,

    I too wish to have a retention range of 6 days.

    DPM is a little different to conventional tape only scheme.

    In DPM, tape is a last resort recovery ( the dpm server is gone lost/stolen/damaged in fire).

    This is were tape steps in as we can recover all of the servers with the tape backup.

    As far as recovering files, I have set a 30 day retention on the dpm server for short term, so recovering files is no problem here. 

    if the dpm server is lost, then off course I can't recover any file version thats not on tape.

    But you have to compromise between cost and protection.

    if I wanted more protection I can have a secondary DPM server etc...

    But the scheme I wanted is using Disks a libraries using firestreamer.

    I have 8 disk , Monday,Tuesday,Wednesday,Thursday, Friday1,Friday2 Friday3,Friday4

    so I want 6 day protection on m,t,w,t and 3 week protection on Fridays.

    If someone wants to recover a file I will use short term protection.

    The Tape-disks are simply last resort disaster recovery.


    pp
    Monday, September 13, 2010 5:37 AM
  •  

    I would echo the request here for a 6 day retention period.  In the vast majority of SME sites daily tape backups are still a must. 

    In lieu of this function and in light of Mike's answer above - Is there a resource anywhere which explains what's required to configure daily long term tape backup with a single PG & a stand alone drive with DPM 2010?   i.e. to use the product to provide the same level of tape backup as the traditional product customers typcally remove when they go DPM?

    * I am talking here about daily long term to tape - no issues with short term to disk - it's great *

    What I'd like to do is pretty much as outlined above by the OP.  A customer typically used 21 tapes with AN Other traditional tape product to provide a year's worth of backups:

    - 4 daily tapes

    - 5 weekly

    -12 monthly

    With DPM we can usually get away with a single PG.  Is there a resource which describes how to achieve this whilst satisfying DPM's tape rotation requirements around the minimum retention period of 1 week?

    I hope this question doesn't sound very negative - I really want to be able to pitch DPM without reservation but it does seem to take a bit of work to maintain daily tape backups.  If there's an easy way to achieve this I would be very glad to hear about it.

     

    Thanks

    ZZZ

     

     

     

     

    Monday, September 13, 2010 10:43 AM
  • The way we worked around this was by creating a second set of Monday to Friday tapes.

    We now have Monday (1) tape and a Monday (2) tape, a Tuesday (1) and a Tuesday (2) tape, etc.

    We set the long term retnetion period for these tapes to be 1 week and rotate the set of daily tapes on a weekly basis. This results in a 2 week gap before reusing the Monday (1) tape again.

    We then have our normal monthly and end of year cycles.

    This solution gets us by the issue of a minimum retention of 1 week by using only 5 more tapes.

    Hope this is clear and helps some people out.

    JP

     

     

    Monday, September 13, 2010 11:02 AM
  • JP

     

    Thanks for your reply, that sounds like a very practical way to get around this.

     

    ZZZ

     

     

    Monday, September 13, 2010 12:01 PM
  • Hmm.  1 tape, who said anything about 1 tape?  Yikes! that would be crazy, I have 13.

    • 4 FireStreamer virtual tapes labeled Mon, Tues, Thur, Fri with a weekly retention (which, in my world, means I want to use them next week)
    • 3 FireStreamer virtual tapes with a 3 week retention (which means I want to use them in 3 weeks, not 3 weeks + one day)
    • The rest rotated monthly

    I've developed some SQL update statements to achieve a common sense notion of tape expiry and will release them after I have observed them in operation for another week.

    With respect to the idea that I would use the tape management report: that's a waste of time which I don't have.  I schedule a year's worth of rotation with our off site media storage company and then assign a non-IT grunt to unplug and plug the drives according to the schedule.  This makes the process so easy that non-technical staff can do it (useful when there is only one full-time IT person).  And don't try to sell me the idea that I should use a script which marks all tapes free, that defeats the purpose of the retention policy enforcement in the software.  If the wrong tape is in the drive due to human error I don't want that tape overwritten.  I expect the software to advise me of the exception condition.

    It appears the software is designed to waste my time.  If I want to change something on a PG I have to click Next, Next, Next, Next so many times that I lose count.  No support for off site hard drive media except through a third party?  Ouch!

    If I want to trigger a tape backup on a PG, why do I have to select each protected item individually?  Who has time for this?  Sigh.

    Wednesday, September 29, 2010 4:40 PM
  • Single PG?  That doesn't sound practical.  You don't have a combination of databases, virtual machines and file servers?

    I agree that DPM should be pitched, but I suspect we are using a different sense of the word.  A bit of work to maintain!  Good one, very funny.
    Wednesday, September 29, 2010 4:45 PM
  • /*
    2010Sep17 Oliver Wilcock: script for DPM2010 to expire tapes 24 hours earlier than DPM would expire them.
    -- This is useful for weekly rotations and other common sense interpretations of retention periods.
    -- Tapes must be online to be affected by the script.
    -- After this script the UI will not show the affected tapes as expired, but the tape engine appears to recognize them as expired and will use them.
    -- To make the UI show the tapes as expired you must restart the UI or trigger a fast inventory. A FastInventory can by triggered with PowerShell like this:
    -- get-dpmlibrary -DPMServerName backup1 | where {!$_.IsOffline} | Start-DPMLibraryInventory -fastInventory
    
    Disclaimer: This script seems to work for me, but I'm making no warranties or claimed fitness to your purpose.
    Known problem: The choice of 0600 for the expiry dates is to accomodate UTC -0500 timezones with DST and my imperfect understanding of how to manipulate dates using Transact SQL.
    Install:
    0. Backup your database. You'd better know how to restore it, because, you know, DPM doesn't know how to do that. Doh!
    1. Create an SQL Agent task and schedule it for a suitable time before your tape backups begin and when your media will be loaded. I use 18:10.
    2. Set the owner of the task appropriately. I used the local account created by the installer MICROSOFT$DPM$Acct.
    3. Create a step, set the type to Transact-SQL, set the database to the DPM database (ie DPMDB), paste this code as the only step.
    4. Test it.
    5. Curse proprietary, closed source, over priced, feature poor, time wasting, software but be thankful for the rare times that it can be hacked, cobbled together and futzed-with-into-the-wee-hours-for-weeks-on-end into a small fraction of a make-shift solution.
    6. Whew!
    
    */
    begin transaction
    update 
     tbl_MM_PhysicalDataset
    
     -- the the ExpiryDate to today at 0600
     set ExpiryDate = DATEADD(HOUR,6, convert( datetime,convert( date, GETDATE() ) ) )
     , LifeStatus = 2
     WHERE
     Datasetid in
     (
    select Dataset.Datasetid
     -- , barcodevalue
     FROM  dbo.tbl_MM_Media AS Media INNER JOIN
     dbo.tbl_MM_ArchiveMedia AS ArchiveMedia ON ArchiveMedia.MediaId = Media.MediaId INNER JOIN
     dbo.tbl_MM_MediaMap AS MediaMap ON MediaMap.MediaId = ArchiveMedia.MediaId INNER JOIN
     dbo.tbl_MM_PhysicalDataset AS Dataset ON Dataset.DatasetId = MediaMap.DatasetId
     WHERE
     Location = 1 -- Media is online
     and ExpiryDate between -- Expiring in the next 24 hours. UTC or local timezone?
      convert(date, GETDATE() )and DATEADD(HOUR,24,GETDATE() )
      -- order by barcodeValue
     )
    go
    
    update 
     tbl_MM_ArchiveMedia
    
     set IsOmidChangeNeeded = 1, DatasetsState = 3
     WHERE
     MediaId in
     (
     select ArchiveMedia.MediaId
     FROM  dbo.tbl_MM_Media AS Media INNER JOIN
     dbo.tbl_MM_ArchiveMedia AS ArchiveMedia ON ArchiveMedia.MediaId = Media.MediaId INNER JOIN
     dbo.tbl_MM_MediaMap AS MediaMap ON MediaMap.MediaId = ArchiveMedia.MediaId INNER JOIN
     dbo.tbl_MM_PhysicalDataset AS Dataset ON Dataset.DatasetId = MediaMap.DatasetId
     WHERE
      DatasetsState = 2 -- not expired/reclaimed
      and Location = 1 -- Media is online
      and ExpiryDate between -- Expiring in the next 24 hours. UTC or local timezone?
      convert(date, GETDATE() ) and DATEADD(HOUR,24,GETDATE() )
     )
    
    go
    commit
    
    
    • Proposed as answer by Oliver Wilcock Tuesday, October 12, 2010 7:38 PM
    Friday, October 1, 2010 2:07 AM
  • I hate resurrecting old posts but I found this one today after MONTHS of trying various PowerShell scripts, etc. to do something as simple as really forcing a tape free that I know is usable.  THANK YOU OLIVER!

    Kelly

    Friday, June 10, 2011 3:51 PM
  • Why cant i set the Retention Time to 5 Days ore 13 Days? Is there any Reason for the DPM to expire the Tapes only weekly?

    Wednesday, July 27, 2011 3:33 PM