none
Please create a feature like PQ query plan for Power Query RRS feed

  • Question

  • As far as I know, PQ creates some internal presentations for calculating M scripts, reshuffling M queries and making something like tunnels/pipelines to data source from a number of steps, which are described in M scripts.

    E.g. internally

    s0 = #table..,
    s1 = Table.AddColumn(s0,"2", each f(_)) ,
    s2 = Table.RemoveColumns (s1,"1")

    is the same as

    s0 = #table.. ,
    s1 = Table.TransformColumns(s0, {"1", each f(_)})

    Defenitely, it would be great to virtually see the resulting "tunnels", just like SQL programmers can see query plans of SQL queries.
    It would help M developers in many ways:
    1. They could really understand what M constructions are more effective that others. Even if you are an experienced M developer, you cannot imaging the whole tunnel for a query of 10 Queries of 10 - 50 steps in each.
    2. They could really understand when they should stop keep visual representing of step result, and start to hard-code 10-ths folding of M functions in one step in PQ UI.

    Now, from my experience, in case of complex ETL, the time, which you gain due to using functional language & rich concept & functionality of M, will be loosed on boring analysis of the possible variants of tunneling and trying different variants without clues what is really more effective or not.

    I placed this as Idea here as well: Query Plan. Please vote for it, if you feel that it suits your needs!

    A number of my question here would appear if PQ would have such a feature...


    Monday, July 29, 2019 1:13 PM

All replies

  • Hi Andrey, not exactly a query plan, but this upcoming feature might be of interest to you:
    https://docs.microsoft.com/en-us/power-platform-release-plan/2019wave2/cdm-data-integration/query-diagnostics-enhancements

    It will provide more insights into a query, and might help understand what folds and what doesn't.
    Tuesday, August 6, 2019 12:07 AM
  • Hi Jorge!

    Yeah, it's a feature I was dreamed about :-).

    Sunday, August 18, 2019 2:38 PM