locked
get data from a dynamic web page loaded through javascript RRS feed

  • Question

  • hello,

    is there any solution to load data from a dynamic webpage that loads most of its elements through javascript (client side)

    there is no user interaction required just the page has a very simple html and the rest of the elements including the tables to extract with Power Query are loaded on accessing the page

    thank you

    Sunday, December 7, 2014 11:39 AM

Answers

  • The existing functionality in Power Query does try to wait until all the scripts on the page have finished running before it takes a snapshot of the DOM. As you might imagine, there's no particularly good way to figure this out reliably, so we have heuristics which work in most -- but obviously not all -- cases.

    When you view the site in the browser, how long does it take for the data to be populated dynamically?

    • Marked as answer by hobbitalex Tuesday, December 9, 2014 5:19 PM
    Monday, December 8, 2014 4:40 PM

All replies

  • The existing functionality in Power Query does try to wait until all the scripts on the page have finished running before it takes a snapshot of the DOM. As you might imagine, there's no particularly good way to figure this out reliably, so we have heuristics which work in most -- but obviously not all -- cases.

    When you view the site in the browser, how long does it take for the data to be populated dynamically?

    • Marked as answer by hobbitalex Tuesday, December 9, 2014 5:19 PM
    Monday, December 8, 2014 4:40 PM
  • Thank you for the info;

    The site, counting from first access takes approximately 3-4 seconds to load in browser all the page elements,

    Maybe in a future update to Power Query, would be great to add an optional parameter to Web.Contents() function where can be specified a custom wait time

    Monday, December 8, 2014 10:55 PM