none
process only chnages

    Question

  • i have a fact table, that gest loaded every day using change data capture, that is every data , the changes that happen in the source are reflected in the destination.

    in the cube i want to be able to process only the changes, what is teh best way to acmplish this

    if i partition by the dateofextract, it does not work since, the line items can move between multiple partitions since change can happen , please guide, is holap a good option to consider.

    thanks

    nik

    Thursday, October 31, 2013 7:05 PM

Answers

  • So if you are updating fact records and can't use the approach of generating your own delta records then you have no choice but to read the entire fact table each time you process.

    Then it simply comes down to tuning your relational queries so that they run faster and/or doing more in parallel. Doing more in parallel is where partitioning may help if you are using SSAS Multi-Dimensional as it can process more than one partition in parallel from the same measure group (tabular cannot do this yet)

    I would suggest reading this whitepaper for details about how to tune the performance of your processing operations http://www.microsoft.com/en-au/download/details.aspx?id=17303


    http://darren.gosbell.com - please mark correct answers

    Sunday, November 03, 2013 11:29 PM
    Moderator

All replies

  • Incremental processing in SSAS can only ever add new fact records. There is no way to update existing facts. So if your ETL updates old facts then you can't do incremental processing. Unless you can compromise by generating detla records.

    eg.

    Old fact value = 100

    New fact value = 90

    So create a delta record of -10

    But this may not be possible in a lot of situations, specially if you have a mixture of measures with different row counts or if have a row count measure.


    http://darren.gosbell.com - please mark correct answers

    Friday, November 01, 2013 4:55 AM
    Moderator
  • So , Can you pls advice on techniques to enhance proceesing speeds, we have huge cubes and partitioning wont help because, data in ole parttions also change, requiring all partitions to be processed, what wud be the ideal techniques to enhance processing speed
    Sunday, November 03, 2013 5:16 AM
  • So if you are updating fact records and can't use the approach of generating your own delta records then you have no choice but to read the entire fact table each time you process.

    Then it simply comes down to tuning your relational queries so that they run faster and/or doing more in parallel. Doing more in parallel is where partitioning may help if you are using SSAS Multi-Dimensional as it can process more than one partition in parallel from the same measure group (tabular cannot do this yet)

    I would suggest reading this whitepaper for details about how to tune the performance of your processing operations http://www.microsoft.com/en-au/download/details.aspx?id=17303


    http://darren.gosbell.com - please mark correct answers

    Sunday, November 03, 2013 11:29 PM
    Moderator