none
SSDT 2015 Partial Cache Lookup transformation taking too long to execute

    Dotaz

  • Hi All,

    While doing partial cache in Look up transformation it takes too long to complete. To find corresponding keys in dimension table I am using look up transformation. Since source data set has millions of records and reference table has also millions of records, Full cache takes all the memory in the server and partial cache takes too long to execute . Can any body suggest a way to fix this issue.

    Thanks

    13. července 2018 7:24

Všechny reakce

  • Why not use a staging table to dump the source data and then do the lookup in database itself using sql query in that case?

    Please Mark This As Answer if it solved your issue
    Please Vote This As Helpful if it helps to solve your issue
    Visakh
    ----------------------------
    My Wiki User Page
    My MSDN Page
    My Personal Blog
    My Facebook Page

    13. července 2018 7:33
  • Thanks for the response Visakh. That's my last option to do it in SQL. Is there a way in Look up we can tweak it and make it work.

    Thanks

    13. července 2018 8:01
  • Hi CoolDbGuy,

    Try to increase buffer size and give proper path to BufferTempStoragePath in Data Flow Properties.

    Regards,

    Pirlo Zhang 


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    16. července 2018 8:40
    Moderátor