Data Warehouses of 15-20 TB using SQL Server


  • Hi,

    Can SQL Server 2012 be used for data warehouses as large as 15-20 TB? We are not considering PDW as yet, but a high end server with Windows Server 2012 is what we have in mind. My questions are:

    Have you created DWs of such high volumes using SQL Sever 2012?

    What are the key areas to focus on while building such high volume DW? Partitions, Parallelism while implementing ETL?

    We are also wondering if breaking the DW into multiple databases would be a good option. Are there any best practices around managing conformed or shared dimensions?

    Request you to please share any suggestions/experiences/best practices.

    Thursday, June 20, 2013 9:20 AM


  • The first challenge is getting a server and storage solution that works well for large data warehouses.  It's extremely tricky to build one from scratch, and extremely expensive to get the multiple GBs throughput you'll need for a good solution from a traditional enterprise SAN.

    So definitely look at the Fast Track Reference Architecture  solutions.  These are cost-effective Data Warehouse solutions engineered in partnership between the SQL Server Data Warehouse team and the hardware vendors.  They are "balanced" systems having the right amount of CPU, memory and storage throughput for a SQL Server Data Warehouse solution.

    Having a DW platform that can scan a very large table quickly, and having the CPU power to process large amounts of data makes all the downstream design decisions easier.



    Thursday, June 20, 2013 1:58 PM