none
Result Caching Problem RRS feed

  • Question

  • Hi guys,

    I have a good one for you! We have deployed FAST in a particularly lively environment such that adding records and queries frequently happen within a short time of each other.

    The problem is as follows:

    1. I ask FAST for records 1-50 for query Q. I examine record number 3 for example and it is X.

    2. I run query Q again but this time I ask FAST for just record number 3 (i.e. start=3, length=1) but upon examination it is not X as expected!

    3. This only happens directly after adding new records to a SharePoint list that FAST crawls.

    4. It seems that record n+1 (so in the example record 4) tends to actually be the X I am looking for in these situations.

    Conclusion: There is some sort of off by 1 caching issue in FAST.

    Could anyone help?

    Many thanks!

    Emir

    Thursday, January 24, 2013 1:53 PM

All replies

  • Hi,

    How many search rows are in your system? It could be that either query goes to a different row which is not 100% in sync with the primary indexer?

    Also, is this a theoretical caching issue or a real issue. If a real issue, what is the use case?

    As long as indexing occurs the index can very well change between query 1 and 2 as the index is constantly being updated, and in this case not a bug the way I see it.

    Thanks,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/MCT/MCPD - If you find an answer useful, please up-vote it.
    http://techmikael.blogspot.com/
    Author of Working with FAST Search Server 2010 for SharePoint

    Saturday, January 26, 2013 8:31 PM
  • Hi Mikael,

    I think we have 2 search rows (will correct post as soon as admin comes back to me if otherwise). The caching issue is real and serious. It occurs when we add some documents to a SharePoint list and they are crawled for the first time. 

    Let's say there are 50 results, for query Q the system retrieves 1-50. When a user clicks on say the 3rd result the system then does the same query Q but starting at index 3 for length 1 and displays that result. However, when new documents have been added these two queries sieze to correspond for some time (5-10 minutes).

    E.g. if we add 20 new files (and then stop adding files) this bug will be present for 5 minutes or more before it just disappears of its own accord. We've already checked our own internal caching.

    Surely, disparity between 2 search rows is a bug, unless the content has changed between queries (which it doesn't in our case)?

    Thanks,

    Emir

    Monday, January 28, 2013 11:37 AM
  • Hi,

    Just want to get the order of acts correct.

    1. You index new items and the crawl finished

    2. You query for items 1-> and look at item 3

    3. You query for items 3-> and the first item is not equal to the one in 2), and no indexing has happened inbetween right?

    4. You wait 5 minutes, then go back to 3) and now the item is correct

    Is this correctly assumed? As for disparity between search rows this is not a bug. Items are first added to the primary row, then synchronized to the other rows. Usually this happens fairly quick and is not a real issue. When debugging, it would be interesting if you could perhaps use my Query Logger tool on both rows and see if query from 2) and 3) are sent to the same row or not.

    Thanks,
    Mikael Svenson 


    Search Enthusiast - SharePoint MVP/MCT/MCPD - If you find an answer useful, please up-vote it.
    http://techmikael.blogspot.com/
    Author of Working with FAST Search Server 2010 for SharePoint

    Tuesday, January 29, 2013 7:34 AM
  • Hi Mikael,

    Thanks for your reply.

    Yes, that order is exactly right. I'll have a look with your tool and see what happens.

    I've implemented a way around this problem by simply not making particular queries for items and instead making the same query regardless of how many rows I need, then caching my way around other problems.

    If synchronisation between rows can take as much as 5 minutes this is quite problematic. If we had a constant (or regular enough) influx of documents we'd have constant discrepancy in this scenario. Further, I'd imagine more search rows would only widen this discrepancy.

    I'll have a look with your tool and see what I can find out. Thanks again for your help.

    Emir

    Tuesday, January 29, 2013 10:26 AM