none
Datawarehouseing and continuous loads (online/real time data)

    Question

  • Hi,

    We have a large datawarehouse with fact-tables (marts) and dimensions which is accessed through database views with Business Objects.
    We load our DW hourly but only update our datamarts every night.
    Now we are getting requests for more online data in our marts (not just every night) and we are having some trouble coming up with a good solution.

    Is there any datawarehousing best practices for more online data and not just one batch every night - even while users are possibly running queries?

    Our marts contain upwards of 500 million rows each and are partitioned. When we load our facts we first do a delete from the fact on all the affected rows and then we re-load affected and new rows to the table.

    Is there any guidelines or best practices for this?
    Any tips on how to achieve more "online" data even with users (possibly) running queries?

    We have some ideas but none of them are really solid.

    Thanks

    Jonathan
    lundi 16 avril 2018 14:03

Toutes les réponses