none
MS DPM Team Supported De-Duplication Products/Vendors RRS feed

  • Question

  • Would someone from the MS DPM Team please respond with and supported de-duplication technologies.  Also, is MS working with BitWackr and/or  BridgeSTOR to provide a DPM Tested and Approved (maybe supported) De-Duplication Solution.

    Monday, November 14, 2011 3:32 PM

Answers

All replies

  • Hardware de-dupe solutions can be used to shrink the storage footprint for DPM.  Virtual tape libraries from vendors like Data Domain (EMC), Quantum and others can be used to de-dupe long term backups.  In addition, the short term (on disk) retention can be decreased if the VTLs are used more frequently.  For instance, if they are saving 30 days in the DPM storage pool, they could shrink that to 15 days and write out more frequently to the VTL.

    DPM team has been working on multiple fronts to provide Dedupe solution for its customers.  As a priority, DPM 2012 is being tested with the following two hardware DeDupe Providers. 

    BitWackr:  
    BitWackr is a Dedupe controller from exar.com and is compatible with Microsoft applications as mentioned in http://www.exar.com/Common/Content/Default.aspx?id=7472

    Also BridgeStor has a DPM appliance with Dedupe built-in support that uses BitWackr in its appliance.  White paper and other information can be located in the below mentioned location.

    • http://bridgestor.com/microsoft-dpm-whitepaper-pdf.html
    • http://www.enterprisestorageforum.com/continuity/article.php/3921956/BridgeSTOR-Debuts-Appliance-for-Microsoft-DPM.htm
    • http://www.eweek.com/c/a/Midmarket/BridgeStor-Unveils-Appliance-to-Bring-Deduplication-and-Compression-to-Microsoft-DPM-179426/


    NetApp Appliance:  Microsoft partnered with NetApp appliance that provides Dedup and compression technologies. 

    Another good option is the Netapp storage arrays which include de-dupe.  This storage can be used by DPM directly for short term retention.  If you a large enough size to warrant a storage array, regardless of de-dupe support, I would recommend looking at using thin provisioning to save space.

    Hope this helps.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, November 15, 2011 4:33 AM
    Moderator
  • Hi to Mike and Marc.

     

    I am the product and product marketing manager for the Exar BitWackr product, the hardware/software combination referred to in the response to the original query. A number of DPM users have contacted me either directly or indirectly expressing interest in our DPM deduplication, compression and encryption technology. We are typically able to help them.

    It would be of interest to me to establish a line of communication with my product and product marketing counterparts at Microsoft and was hoping you might be able to give me some assistance and direction in that.

    Thank you and Happy Holidays.

    Bob

    (I can be reached at my business email of bob.farkaly@exar.com) 

    Wednesday, December 14, 2011 7:08 PM
  • Hi Mike,

    In reply to your post on November 14th, BridgeSTOR sells the bitwacker card and software as well as the appliances.  Exar is an OEM company and will sell bitwacker cards of 100 quantity or more.  BridgeSTOR is the marketing and sales for the card and software.  I am happy to help end users in their quest for deduplication for DPM!  See  www.bridgestor.com

    Thanks for the mention!

    Dawn

    dmatze@bridgestor.com

    Monday, December 19, 2011 11:15 PM
  • Hi,

    we have tested Datadomain and FTS CS800 (I think it is quite similar to a Quantum but I am not sure) for DPM.

    If you have questions regarding those  I could perhaps give answers...

     

    regards

    /bkpfast

    Tuesday, December 20, 2011 2:40 PM
  • Is it safe to say then that the Bridgestor DPM 2010 card will work with DPM 2012? Also it is safe to assume that DPM 2012 will provide no native dedup algorithms? 

    Paul

    Tuesday, January 10, 2012 2:45 PM
  • Hi Vision20,

    Yes, DPM 2012 will work with the Bridgestor card and like DPM 2010, DPM 2012 has no native dedupe feature.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, January 10, 2012 4:36 PM
    Moderator
  • hi bkpfast,

    we are currently evaluating a CS800 for a DPM project. my problem, I don't know much about the workings and concepts of DPM. from what I heard DPM has pretty IO intensive maintenance jobs doing reorganization does it even make sense to use a dedupe system as a first tier backup target? how does the CS800 (or any) dedupe system cope with the reorganization jobs etc.?

    many thanks for your inputs,
    sean

    Wednesday, January 18, 2012 1:31 PM
  • Hi sean,

    the problem you will get into is that the CS800 can only provide inline DeDup when using CIFS, NFS or a VTL (Virtual Tape Library).
    For first tier backup you would usually use a (local visible) disk / LUN so that is what the CS800 (the same for Datadomain) cannot provide.

    You could do a what DPM calls "Short Term Protection to Tape" and use the VTL then but I think it is not the recommended way...

    The problem we had (have) is that when DPM backs up to Tape (VTL) it creates a snapshot on the required volumes (locally), mounts it, and does a full write to Tape.
    When you have got several file servers with millions of files copying to tape consumes a lot of ressources of the DPM Servers OS.
    We did not have performance issues on the CS800 but DPM just didn't deliver data fast enough although the primary disk storage (Tier1) was an exclusivly used EMC Clariion CX3-40.
    The thing we learned is: Not too many tape jobs at the same time and enough RAM (>32 GB).

    But probably we are going to purchase some CS800 because the dedup ration is quite impressive (with the new firmware and inline dedup!!)

    We did a lot of testing regarding disk performance.

    When you gotfurther questions - feel free

    regards


    /bkpfast

     

    Wednesday, January 18, 2012 3:49 PM
  • Hi.

    Does anyone have any experience with Microsoft DPM according to EMC DataDomain Vs Bridgestor?

    Regards

    Hem

    Monday, January 23, 2012 8:53 AM
  • My experience with using the bridgestor cards, We only roughly got about 20% decrease in overall storage amounts, however you can store more recovery points on disk and the storage savings are slowly increasing. We had a bit of trouble configuring and getting it going and there are a few caveats we did encounter...but was eventually sorted, however im located in Australia so support/supply of the cards is a bit complicated.

    However as Mike pointed out before about using a VTL they seems to be alot more functional. We ended up keeping dailys on DPM for 2 weeks, weeklies to VTL for 1month, Monthlys to  the VTL for 12months and Yearly to the VTL for 7 years, we then use the replication on the VTL for an offsite copy. As you can see we didnt write to tape at all. The downside with most of the DPM VTL solutions we found was if you wanted to go back to tape it had to go back through DPM, and DPM has quite average tape performance (eg no multiplexing etc...) when your talking about 40TB of data that matters and was a choking point for us.

    As usual depends on your requirements, but i hope thats somewhat hopeful.

    Monday, February 13, 2012 5:29 AM
  • Hi,

    What VTL solutions are you using with DPM that supports replication. Any information on this will be appreciated.

    Wednesday, February 15, 2012 12:06 PM
  • Quantum DXI6700 and Dell/EMC DD670

    We were using 2 solution due to a company athat was aquired, both perform exactly the same. Quantum is cheaper, EMC is alot more scalable and easier to setup

    Full specs can be found on their websites

    Wednesday, February 15, 2012 12:37 PM
  • Hi,

    Can you please let us know your experience with Data Domain and DPM. What compression ratio were you getting? We evaluated their device a couple of years ago with BackupExec and we were getting about 20x (20:1) ratio. Did you get same ratio? What was the performance?

    Please let me know your recommendations of using it with DPM or do you recommend not using it with DPM?

    Thanks

    Thursday, February 16, 2012 8:13 PM
  • Hi DanoARX,

    we tested Datadomain and CS800S2 and both had good ratios - it was about 20-24:1.

    It was good for DPM but as mentioned before (I guess...) when you got big file servers with a lot of files DPM To tape takes some time but that is not the fault of the VTLs...

    regards

    /bkpfast


    My postings are provided "AS IS" with no warranties and confer no rights

    Friday, February 17, 2012 12:30 PM
  • FYI, Bridgestor is no longer selling the De-Dupe card. I contacted them to purchase one of the cards and they stated they are moving to an all software solution that will focus on the Virtual environments first. The email and accompanying PDF did nto say anything about a time frame for DPM. 

    Anybody know where else to by a Exar card or do we have to wait for Windows Server 2012 and DPM 2012 SP1?

    Wednesday, June 13, 2012 12:56 PM
  • Having the same doubts about DPM and dedupe. It seems DPM 2012 R2 understands the Dedupe, but what if I want something more generic. Do I have to go with bridgestor or is there any list of HW "dedupers" that will potentially see the raw DPM volumes?

    Jose


    MCITP, MCSE, MCTS

    Monday, November 18, 2013 7:55 PM
  • Just to add some further detail, Bridgestor has now stopped shipping their Crunch product, in favor of their cloud ready Coronado offering. Not really an option for us, since we have so much data to protect, putting it all into Amazon S3 or similar would be cost prohibitive.

    Here is to hoping that Microsoft will see the light and add deduplication to DPM, and make it the leading backup product on the market!

    Thanks,

    Karl

    Tuesday, November 26, 2013 2:58 PM