none
Privacy settings impact on query execution RRS feed

  • Question

  • Hello everyone,

    I have a number of queries that fetch data from difference data sources (web, Azure SQL, Azure Table Storage) and some others that merge all the data together. The way I understand what I made, my Azure Table Storage data source should only be hit once due to caching (the connector uses the REST API of the storage service under the hood). This is exactly what I see in Fiddler when I hit the Refresh button with the privacy settings set to Ignore:

    GET http://worldtimeapi.org/api/ip
    200 OK (application/json)
    
    CONNECT http://********.table.core.windows.net:443
    200 Connection Established ()
    
    GET https://********.table.core.windows.net/telemetry?$filter=PartitionKey%20eq%20'****'%20and%20Timestamp%20ge%20datetime'2020-03-31T22:00:00'%20and%20Timestamp%20le%20datetime'2020-07-29T09:35:31.277691'&$select=RowKey,Body
    200 OK (application/atom+xml)
    
    GET https://********.table.core.windows.net/telemetry?%24filter=PartitionKey%20eq%20%27****%27%20and%20Timestamp%20ge%20datetime%272020-03-31T22%3A00%3A00%27%20and%20Timestamp%20le%20datetime%272020-07-29T09%3A35%3A31.277691%27&%24select=RowKey%2CBody&NextPartitionKey=********&NextRowKey=****************
    200 OK (application/atom+xml)
    
    GET https://********.table.core.windows.net/state?$filter=PartitionKey%20eq%20'****'&$select=RowKey,DeviceType,FirstUpdated,LastUpdated,BatteryLevel,BatteryLevelLastUpdated,Location,LocationLastUpdated,AccurateLocation,AccurateLocationLastUpdated
    200 OK (application/atom+xml)

    However, when I change the privacy settings to Combine and set the privacy level of all data sources to Public, the table storage starts being hit multiple times per refresh and I don't understand the cause of the extra queries. Specifically, the unexpected additional queries have $top=1000 at the end which typically corresponds to preview queries, though these never happen with privacy settings set to Ignore (I also forbid the download of data previews in the background in the settings):

    GET http://worldtimeapi.org/api/ip
    200 OK (application/json)
    
    CONNECT http://********.table.core.windows.net:443
    200 Connection Established ()
    
    GET https://********.table.core.windows.net/telemetry?$top=1000
    200 OK (application/atom+xml)
    
    GET https://********.table.core.windows.net/telemetry?$filter=PartitionKey%20eq%20'****'%20and%20Timestamp%20ge%20datetime'2020-03-31T22:00:00'%20and%20Timestamp%20le%20datetime'2020-07-29T09:45:55.268871'&$select=RowKey,Body
    200 OK (application/atom+xml)
    
    GET https://********.table.core.windows.net/telemetry?%24filter=PartitionKey%20eq%20%27****%27%20and%20Timestamp%20ge%20datetime%272020-03-31T22%3A00%3A00%27%20and%20Timestamp%20le%20datetime%272020-07-29T09%3A45%3A55.268871%27&%24select=RowKey%2CBody&NextPartitionKey=********&NextRowKey=****************
    200 OK (application/atom+xml)
    
    CONNECT http://********.table.core.windows.net:443
    200 Connection Established ()
    
    GET https://********.table.core.windows.net/telemetry?$filter=PartitionKey%20eq%20'****'%20and%20Timestamp%20ge%20datetime'2020-03-31T22:00:00'%20and%20Timestamp%20le%20datetime'2020-07-29T09:45:55.268871'&$select=RowKey,Body&$top=1000
    200 OK (application/atom+xml)
    
    CONNECT http://********.table.core.windows.net:443
    200 Connection Established ()
    
    GET https://********.table.core.windows.net/state?$filter=PartitionKey%20eq%20'****'&$select=RowKey,DeviceType,FirstUpdated,LastUpdated,BatteryLevel,BatteryLevelLastUpdated,Location,LocationLastUpdated,AccurateLocation,AccurateLocationLastUpdated
    200 OK (application/atom+xml)

    My question is why am I seeing a difference at all? I would have assumed that if the privacy level of all data sources is set to Public, the behavior would be exactly the same as when the privacy levels are ignored.

    Cheers,

    Elliot



    Wednesday, July 29, 2020 5:30 AM

All replies