Multi thread advice needed


  • Hi All,

    I have 50+ files to be loaded to SQLServer, I've created a table to load the stats and other details for performing this action.

    I am looping through this table using for each loop in SSIS, which has all the details what to load, from where to load, which package to executed, what is the source file/table/etc. what is the target table for each line (source file) etc...

    Each line item in this table is used to load one file.

    I have created a master package in which I am using foreach loop to loop through all the records in this table and load the data from source to target, in foreach loop I am calling the child package to perform this load, I have created N child pkgs where N= No of recs(no of files) in the stats table (N is fixed for the number of files to be loaded)

    I am using package configuration to pass the all the required input to the child pkgs from master pkg for performing the data load, this is working perfectly fine.

    Now the question is, as I am doing this in serial one after the other, the performance  is slow which is taking lot of time. I am looking for bringing in multi threading in this, I mean to say execute multiple files in parallel ( at least 3-4 files in parallel) can any one help me in redesigning this to bring in parallelism to improve the performance.



    Tuesday, August 5, 2014 5:26 AM


All replies