Sign in
 
Home20122008 R2200820052000LibraryForums
SQL Server >
> 
> 
Loading data that comes in every second of every day

全般的な情報交換 Loading data that comes in every second of every day

  • Tuesday, November 27, 2012 3:37 PM
     
     
    Sign In to Vote
    0

    We are looking for something to help us load LARGE amount of data ... streaming in EVERY second.  Is streamlight deisgned for this or do we need to look at another product.  We use SSIS already for our data loads and would like to stay in the MS tools / SQL Server space.

    Please advise

     

All Replies

  • Thursday, November 29, 2012 8:57 AM
    Moderator
     
     
    Sign In to Vote
    0
    Hi RichardB_DBA ,

    To provide fast data insert operations, SQL Server ships with several of standard bulk load methods. 

    1. Integration Services Data Destinations – The method for performing bulk load from the Integration Services ETL tool.
    2. BCP – The command line utility for performing bulk load.
    3. BULK INSERT – The method for performing bulk load in Transact-SQL
    4. INSERT ... SELECT – The method for performing bulk load in process with SQL Server from local queries or any OLE DB source. This method is only ailable as a minimally logged operation in SQL Server 2008.
    5. SELECT INTO – The method for creating a new table containing the results of a query; utilizes bulk load optimizations. 

    For more information about the data loading performance: please see: http://msdn.microsoft.com/en-us/library/dd425070(v=sql.100).aspx 

    Thanks,
    Eileen

    If you have any feedback on our support, please click here

    Eileen Zhao
    TechNet Community Support

     
  • Friday, December 07, 2012 1:54 PM
     
     
    Sign In to Vote
    0

    Hi Richard,

    How much data do you consider a large amount? You might be able to get away with having a SSIS package that runs in an infinite loop. If the job runs in an infinite loop, you can avoid all the start-up costs involved with running a SSIS package. Since there isn't any information on your environment, I have no idea if it will work for you, but it is the easy option since you are using SSIS already. If you have a test environment, I'd test it out and see if it falls behind. If you really need to cut the overhead of using SSIS, you might need to code it in .Net.

    I haven't worked with StreamInsight, but my understanding is that it is used when loading data into a database is impractical. If you are attempting to load data into your database as quickly as possible without doing any processing on it, StreamInsight is probably not the solution you are looking for.