none
Data Protect Manager - large filestreams RRS feed

  • Question

  • We are currently in the final stage testing DPM 2010 , however i have run into a problem. We have a database with a large number of files in a filestream , 23,000 (about 4gb). DPM fails to create a replica for this database .All of the other databases on this server backup with no issue

    Ihave check all of the permissions and they are all correct.

    Check the database backs up as a normal maintainence job , no issues

    Check VSS admin , Sql writer is stable at first but when i create the a PG for this database  , shortly after the SQL WRITER go to waiting for completion and this status does not change

    Any thoughts on this would be greatly appreciated.

     

    Tuesday, May 10, 2011 8:30 AM

Answers

  • Hi,

    What are the details of the failed DPM job ? 

    A SQL Database having filestream itself is not an issue but if the number of file streams for the database are very high, DPM has an internal 10 MB DCOM limit for agent communications and if exceeded will cause problems.  It's not the size of the files in the data stream it’s the number of file streams that cause us to exceed the 10MB dcom limit.

    Before backing up a database, we pass the backup metadata from the protected server (SQL) to the DPM server and we are passing each file name in the FILESTREAM folder as a node in the metadata Xml file. If the .xml file exceeds 10MB, we will fail. This is planned to be fixed in the next release of DPM.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, May 11, 2011 3:46 AM
    Moderator

All replies

  • Hi,

    What are the details of the failed DPM job ? 

    A SQL Database having filestream itself is not an issue but if the number of file streams for the database are very high, DPM has an internal 10 MB DCOM limit for agent communications and if exceeded will cause problems.  It's not the size of the files in the data stream it’s the number of file streams that cause us to exceed the 10MB dcom limit.

    Before backing up a database, we pass the backup metadata from the protected server (SQL) to the DPM server and we are passing each file name in the FILESTREAM folder as a node in the metadata Xml file. If the .xml file exceeds 10MB, we will fail. This is planned to be fixed in the next release of DPM.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, May 11, 2011 3:46 AM
    Moderator
  • Hi,

    I think we are having the same issue. We have a sharepoint 2010 site using sql server 2008 R2 with filestream. There are currently around 60.000 files in de blob store and DPM can't seem to back it up either via a Farm backup or SQL backup. When you say "This is planned to be fixed in the next release of DPM", do you mean the next hotfix rollup? The last hotfix rollup is I think from march 2011

    Wednesday, May 18, 2011 4:11 PM
  • Hi,

     

    DPM 2012 is planned to contain the fix for this issue.  I don't know the complexity of the fix, so I cannot say if it will be back ported to DPM 2010 or not. 


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, May 18, 2011 4:14 PM
    Moderator
  • Therefore if this is the case then reducing the number of file should make it work. Ok , i testeded it on the lab box.

    Started with 27,000 files , no joy didn't backup

    Deleted 10000 rows  (17000 rows left), deleting the rows should delete the underlying files. Still no joy

    Deleted all but 5000 rows  , still no joy

    Delete all but one row ,guess what still didn't replicate.

    Next step in to recreate the same database from scratch it now data , test it backs up the repopulate it in chunks of 1000 rows till it fails.

    Will keep you posted...

     

     

     

     

    Thursday, May 19, 2011 12:55 PM
  • We have 2 sharepoint sites in a farm running with SQL filestream. Our portal has around 60.000 files in de blobstore, DPM2010 wil not back it up. The other site (Mysite) also has an blobstore with around 4200 items, DPM has no problems with it. The backup only fails on the portal blobstore with 60.000 files. I hope there will be a fix for DPM 2010 also
    Thursday, May 19, 2011 2:45 PM
  • Seniordb,

    The issue may not be related to the file streams, what are the details of the failed jobs ?  You may need to look at the dpm logs to determine the true cause.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, May 19, 2011 6:09 PM
    Moderator
  • Inity , can i ask

    on the database that doesn't backup , have any rows been delete

     

    Friday, May 20, 2011 12:43 PM
  • As far as I know, no rows have been deleted

    Saturday, May 21, 2011 10:03 AM