none
Storing large files in SharePoint 2010

    Question

  •  

    Hi,

    We are developing a portal in SharePoint 2010 in which we need to upload large files.  The files could be audio/video/PPTs/PDFs/SWF etc. The number of files will increase with time. What would be the best option to store these files so that this should not have impact on the performance and scalability of the SharePoint portal.


    Regards, Parveen

    Friday, February 10, 2012 6:39 AM

Answers

  • Actually 200 GB is not a hard limit. Updated sizing info has been published a while ago and the usual limit is now 4 TB:

    http://technet.microsoft.com/en-us/library/cc262787.aspx

    For best performance you might want to enable blob caching and/or use Remote Blob Storage to store the files on the file system rather than in SQL:

    http://blogs.technet.com/b/stefan_gossner/archive/2011/06/23/sharepoint-2010-rbs-documentation-has-been-updated.aspx


    Stefan Goßner
    Senior Escalation Engineer - Microsoft CSS
    This post is provided "AS IS" with no warrenties and confers no rights.

    Friday, February 10, 2012 7:16 AM
  • Hi,

    By default file upload size limit is 50MB. If you want to increase size limit you can do this from central Admin. You may also require to increase Web Page Security Validation time.

    Plz visit

    http://www.anmolrehan-sharepointconsultant.com/2012/01/sharepoint-2010-document-library-upload.html

    • Proposed as answer by Rehan Anmol Saturday, February 11, 2012 4:53 AM
    • Marked as answer by Rock Wang– MSFT Friday, February 17, 2012 3:22 AM
    Saturday, February 11, 2012 4:53 AM
  • I know it's a technicallity, but it's not SharePoint that supports RBS per say, but the back end SQL server.  It is possible to be running SharePoint 2010 and not be able to utilize RBS because your SQL server doesn't support it. As I recall the FILESTREAM provider needs to be installed on each sharepoint server and database server and to run RBS I think you've got to be using SQL Server 2008 R2 Enterprise.

    Also, in response to the original post, if you're going to be streaming a large number of files for unknown quantities of bandwidth/audiences it might be worthwhile to look into either setting up a streaming media server or utilizing a content distribution network.

    It depends on how you utilize the site, if it's internal and for collaboration, i'd keep content and site together.  If it's internet facing/publishing environment then I'd probably at least seperate the very large content/video into a seperate web app and just link to the reference material from the main content site so I'd be able to split it out with load balancer magic to isolate the two web apps if i needed to later on or upsize to a different architecture without to much hassle.

    Tuesday, February 14, 2012 5:18 PM

All replies

  • SharePoint content databases have recommended size of the content database to be below 200 GB so you might have to plan your site and site collection structure that way.


    Varun Malhotra
    =================
    If my post solves your problem could you mark the post as Answered or Vote As Helpful if my post has been helpful for you.

    Friday, February 10, 2012 7:08 AM
  • Hi there,

    the limit for filesize in SharePoint is 2GB per file. To optimize your storage I would definitely recommend to have a separate content database for every site collection. The limit for content database is now increased to 4TB with SharePoint 2010 SP1 http://blogs.msdn.com/b/pandrew/archive/2011/07/08/articles-about-scaling-sharepoint-to-large-content-database-capacity.aspx

    For optimizing the performance and scalability you can utilize remote blob storage
    http://technet.microsoft.com/en-us/library/ee748638.aspx when your files will be stored outside your content database on your filesystem. Also with content dbs you can perform some optimalizations on the SQL server level (separate data/logs, optimize tempdb etc.).


    Marek Chmel, WBI Systems (MCTS, MCITP, MCT, CCNA)
    Please Mark As Answer if my post solves your problem or Vote As Helpful if a post has been helpful for you.

    Friday, February 10, 2012 7:15 AM
  • Actually 200 GB is not a hard limit. Updated sizing info has been published a while ago and the usual limit is now 4 TB:

    http://technet.microsoft.com/en-us/library/cc262787.aspx

    For best performance you might want to enable blob caching and/or use Remote Blob Storage to store the files on the file system rather than in SQL:

    http://blogs.technet.com/b/stefan_gossner/archive/2011/06/23/sharepoint-2010-rbs-documentation-has-been-updated.aspx


    Stefan Goßner
    Senior Escalation Engineer - Microsoft CSS
    This post is provided "AS IS" with no warrenties and confers no rights.

    Friday, February 10, 2012 7:16 AM
  • Hi,

    By default file upload size limit is 50MB. If you want to increase size limit you can do this from central Admin. You may also require to increase Web Page Security Validation time.

    Plz visit

    http://www.anmolrehan-sharepointconsultant.com/2012/01/sharepoint-2010-document-library-upload.html

    • Proposed as answer by Rehan Anmol Saturday, February 11, 2012 4:53 AM
    • Marked as answer by Rock Wang– MSFT Friday, February 17, 2012 3:22 AM
    Saturday, February 11, 2012 4:53 AM
  • Parveen that is a great question, it should be open for discussion which is the best way of handling large amount of data in SharePoint/SQL - its religion :)

    BLOB would off course be the first answer - and a good one, but it also depends on you environment especially your SQL backend, storage etc., MS best practice or recommendations is not YES or NO.

    Take a look what MS states about BLOB: http://technet.microsoft.com/en-us/library/ff628583.aspx

    My favourite http://www.metalogix.com/Products/StoragePoint.aspx - again someone alse would say no use ????????????? :)

    So you have to do a litle research in regards what would be the best solution in your environment, you can choose to keep data in SQL or unload via BLOB, both scenarios have pro and cons.  

    And not to forget the files the audio/video is that for streaming, are they going to be accessed, or is it for archiving.
    Saturday, February 11, 2012 2:13 PM
  • Hi,

    Thanks to all of your for providing the inputs here. Our requirement is that user should be able to view the video files. The video file should start playing once user clicks on the same. User should not wait for complete video to be downloaded to user's machine.

    I hope this clarifies the video requirements here.


    Regards, Parveen

    Tuesday, February 14, 2012 4:15 PM
  • SharePoint has a facility known as RBS (Remote Blob Storage) that allows you to store the metadata within the Content Databases whilst moving the actual content to cheaper forms of storage.  This can be both programmed and catered for via a third party product.  It's bene mentioned a few times already but it's important to realise that you'll either need a developer for it or a third party tool.

    As for the video requirements, that's a little basic.  An AVI file will be much larger than a SWF showing exactly the same thing, so the file formats will need to be given a lot of consideration.


    Steven Andrews | SharePoint Professional | http://www.twitter.com/backpackerd00d | https://baron72.wordpress.com/

    Tuesday, February 14, 2012 4:45 PM
    Answerer
  • I know it's a technicallity, but it's not SharePoint that supports RBS per say, but the back end SQL server.  It is possible to be running SharePoint 2010 and not be able to utilize RBS because your SQL server doesn't support it. As I recall the FILESTREAM provider needs to be installed on each sharepoint server and database server and to run RBS I think you've got to be using SQL Server 2008 R2 Enterprise.

    Also, in response to the original post, if you're going to be streaming a large number of files for unknown quantities of bandwidth/audiences it might be worthwhile to look into either setting up a streaming media server or utilizing a content distribution network.

    It depends on how you utilize the site, if it's internal and for collaboration, i'd keep content and site together.  If it's internet facing/publishing environment then I'd probably at least seperate the very large content/video into a seperate web app and just link to the reference material from the main content site so I'd be able to split it out with load balancer magic to isolate the two web apps if i needed to later on or upsize to a different architecture without to much hassle.

    Tuesday, February 14, 2012 5:18 PM
  • Feel free to ping me if you have not solved this issue at brandong@attunix.com.  My company Attunix has a simple solution to get around this limitation.  Thanks, Brandon
    Monday, November 19, 2012 10:28 PM
  • Hi Parveen,

    Remote BLOB Storage and specifically Metalogix StoragePoint were mentioned previously in this thread as potential solutiions to your performance and scalability concerns. However, you said audio, video and SWF files were part of the mix so I'm wondering if you also need support for files over 2 GB in SharePoint? I'm with the StoragePoint product team and until now I couldn't have helped.

    However, next month StoragePoint v4.1 will be GA with support for files over 2 GB. Feel free to reach out to me at trossi at metalogix if you would like to learn more.

    • Proposed as answer by Tom Rossi Tuesday, February 26, 2013 6:04 PM
    • Unproposed as answer by Steven AndrewsEditor Wednesday, February 27, 2013 11:57 AM
    Tuesday, February 26, 2013 6:04 PM
  • This an old post and I hate to ask a question over 2 years later but here goes. We are considering raising the upload file limit to 250mb. What could be the possible downfalls? 
    Saturday, March 08, 2014 2:43 PM
  • A lot of replies focused on the governance of a huge Content Database that might turn up quickly if uploading many huge files. I shall only warn that if using SQL Server RBS for performance, Microsoft enforces the total size of RBS and content database should not exceed 200 GB. Beyond that, good DBA's will be needed.

    From the file type examples, I see most are files that are only edited once, so when they land at SharePoint they become pretty static. Due to this, I'd suggest to use Asset Libraries and make each item a Asset. Even explore Record Center site template a bit.

    I'm also concerned that you might have a steady number of customers downloading each large video/audio file everytime they click. This has the same bad impact on the web front ends as when they're being uploaded the first time. The IISs get stuck for a few seconds. To work around this impact, I'd suggest some kind of solution of a third party player that would smoothly stream the video/audio file on the browser, using sliverlight, and a IIS's Bit Rate Throttling relaxing the load at the front ends.

    Monday, March 10, 2014 10:26 AM