Answered by:
SharePoint Library for Large Amounts of Engineering Data

Question
-
We are currently using traditional project directory folders for large projects with sometimes tens of thousands of documents. We are planning on migrating the data to SharePoint and the path forward in unclear.
Initially it was recommended to use a library, not numerous folders, to contain the data so that searching of data in improved. That sounded great. The 1<sup>st</sup> project used to pilot this for other project is divided into 20 different modification packages. A library category was created for MODS with selectable options of the 20 mod package names and “No Defined” (default value). Some data items are shared between more than one MOD so this category can have more than one assignment.
When we looked at the directory structure in place we found no consistency in folder names, no consistency in directory structure. Many folders have 5 or 6 (or more) levels of subdirectories. Ideally we want no more than 4 or 5 categories of meta data to define all data. Mapping from chaos into a comparatively small number of categories is daunting.
When searching this forum I find that libraries should be limited to 2,000 items. There are tens of thousands of items in our pilot project. Surely someone somewhere has encountered this organizational problem. I could use some advice from someone who have been there before.
Thursday, February 20, 2014 5:06 PM
Answers
-
John,
The limit of 2000 is not a hard limit, the actual no of items you can store in a list is 30,000,000. however more item would have impact on performance on rendering and lock on the SQL table.
Also the limit that you have mentioned (2000) is list view threshold limit and actually it is 5000.
One important aspect is Boundaries are hard limit, which you cannot exceed and Supported limits are limits based on tests, which can be exceeded but may cause issues.
Being said that , I would suggest you to check out this link on
SharePoint Server 2010 capacity management: Software boundaries and limits
http://technet.microsoft.com/en-us/library/cc262787(v=office.14).aspx
and explore other ways of optimizing your list
here are some references that would help you to optimize -
http://technet.microsoft.com/en-us/library/cc262813(v=office.14).aspx
http://office.microsoft.com/en-us/sharepoint-server-help/sharepoint-lists-v-techniques-for-managing-large-lists-RZ101874361.aspx
Hope this helps!
Ram - SharePoint Architect
Blog - http://www.SharePointDeveloper.in
Please vote or mark your question answered, if my reply helps you- Edited by Ramakrishnaraja Thursday, February 20, 2014 8:09 PM verb confusing
- Marked as answer by Hemendra AgrawalModerator Monday, March 10, 2014 9:07 AM
Thursday, February 20, 2014 8:04 PM
All replies
-
2k is just a suggestion. Note that there is a customizable option for the list threshold, which will default to 5k per container. Container meaning that if you have the entire library of 10k, that's one container. Throw two folders in there and evenly distribute the documents 5k each and you're fine. In that sense, one library is fine as long as you use folders to separate the content. Folders are also great for handling permissions if it may be unique.
That said, if you could split into multiple libraries or site collections, that could be good for future growth and performance.
Andy Wessendorf SharePoint Developer II | Rackspace andy.wessendorf@rackspace.com
Thursday, February 20, 2014 7:49 PM -
John,
The limit of 2000 is not a hard limit, the actual no of items you can store in a list is 30,000,000. however more item would have impact on performance on rendering and lock on the SQL table.
Also the limit that you have mentioned (2000) is list view threshold limit and actually it is 5000.
One important aspect is Boundaries are hard limit, which you cannot exceed and Supported limits are limits based on tests, which can be exceeded but may cause issues.
Being said that , I would suggest you to check out this link on
SharePoint Server 2010 capacity management: Software boundaries and limits
http://technet.microsoft.com/en-us/library/cc262787(v=office.14).aspx
and explore other ways of optimizing your list
here are some references that would help you to optimize -
http://technet.microsoft.com/en-us/library/cc262813(v=office.14).aspx
http://office.microsoft.com/en-us/sharepoint-server-help/sharepoint-lists-v-techniques-for-managing-large-lists-RZ101874361.aspx
Hope this helps!
Ram - SharePoint Architect
Blog - http://www.SharePointDeveloper.in
Please vote or mark your question answered, if my reply helps you- Edited by Ramakrishnaraja Thursday, February 20, 2014 8:09 PM verb confusing
- Marked as answer by Hemendra AgrawalModerator Monday, March 10, 2014 9:07 AM
Thursday, February 20, 2014 8:04 PM