none
What is ImportanceLevelWeight in FullTextIndexRankComponent? RRS feed

  • Question

  • Can anyone explain what is ImportanceLevelWeight in FullTextIndexRankComponent?

    I assume it is the actual weighting given to each importance level in FullTextIndexRankComponent, so the higher the importance level would have higher weight. Then I found out the weight for importance level 1 is 30 in the default rank profile, the same as importance level 4. This is higher than the weight given to importance level 3 (weight=20) and importance level 2 (weight=10). 

    Does this mean text in importance level will get ranked higher than the text in importance level 2 and 3? Hope someone knows the in and out of rank profile can shed some light on this.

    Thanks

    Ben

    Thursday, June 23, 2011 8:28 PM

Answers

  • Hello Ben,

    The idea is that if your search query words match in a field defined with a higher priority, then it will get ranked higher, or given a higher boost. The weight score is a percentage from 0-100 which is relative to the other fields in your composite field.

    The composite field, in this case "content", is the field which contains all the managed properties defined by the index profile.

    So, over to the levels and the scores. I was somewhat puzzled myself that Level 1 had a score of 30, while Level 2 had 10 and Level 2 had 20. From what I can read this means that fields put into Level 1 will in fact rank higher than fields in Level 2 and 3.

    I did some tests on this, and indeed, content in Level 1 will rank higher than Level 2. I did a test with three documents where the content was the same. Doc1 was put in a level 1 mapped managed propery, doc2 in a level 2, and doc3 in a level 5.

    I could see from the rank log on the query that the document with data in level 1 got a higher context score compared to the one with data in level 2. But in the default rank profile the context score is not given that much value compared to other rank components, so all three documents came out with the same final score.

    If I increased the importance of the context score from 50 to 200, then the differences panned out to the final score as well, where doc3 came first, then doc1 and doc2 at the end.

    Conclusion: Someone seems to have messed up the default values.

    The levels also serve a different purpose. That is for the drill levels related to the stop word threshold.

    For the default fulltext configuration the stop word threshold is set to 2.000.000. If you execute a query which will return more than 2 million hits, then it will first drop hits in the lowest importance levels (1+2) when ranking the results. If it still gets 2 million after dropping level 1+1, then it will drop level 3+4, then drop 5+6, until it will get fewer hits than the stop word threshold. Once it has managed to rank the results, it will append all the other results at the end for the levels dropped, but without the field importance ranking.

    Hope this didn't get too complicated. If you take a look at C:\FASTSearch\index-profiles\deployment-ready-index-profile.xml you can see how it's all tied together.

    Regards,
    Mikael Svenson 


    Search Enthusiast - MCTS SharePoint/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    • Proposed as answer by Gunnar Braaten Friday, June 24, 2011 10:49 AM
    • Marked as answer by Ben Liang Friday, June 24, 2011 1:23 PM
    Friday, June 24, 2011 9:25 AM

All replies

  • Hello Ben,

    The idea is that if your search query words match in a field defined with a higher priority, then it will get ranked higher, or given a higher boost. The weight score is a percentage from 0-100 which is relative to the other fields in your composite field.

    The composite field, in this case "content", is the field which contains all the managed properties defined by the index profile.

    So, over to the levels and the scores. I was somewhat puzzled myself that Level 1 had a score of 30, while Level 2 had 10 and Level 2 had 20. From what I can read this means that fields put into Level 1 will in fact rank higher than fields in Level 2 and 3.

    I did some tests on this, and indeed, content in Level 1 will rank higher than Level 2. I did a test with three documents where the content was the same. Doc1 was put in a level 1 mapped managed propery, doc2 in a level 2, and doc3 in a level 5.

    I could see from the rank log on the query that the document with data in level 1 got a higher context score compared to the one with data in level 2. But in the default rank profile the context score is not given that much value compared to other rank components, so all three documents came out with the same final score.

    If I increased the importance of the context score from 50 to 200, then the differences panned out to the final score as well, where doc3 came first, then doc1 and doc2 at the end.

    Conclusion: Someone seems to have messed up the default values.

    The levels also serve a different purpose. That is for the drill levels related to the stop word threshold.

    For the default fulltext configuration the stop word threshold is set to 2.000.000. If you execute a query which will return more than 2 million hits, then it will first drop hits in the lowest importance levels (1+2) when ranking the results. If it still gets 2 million after dropping level 1+1, then it will drop level 3+4, then drop 5+6, until it will get fewer hits than the stop word threshold. Once it has managed to rank the results, it will append all the other results at the end for the levels dropped, but without the field importance ranking.

    Hope this didn't get too complicated. If you take a look at C:\FASTSearch\index-profiles\deployment-ready-index-profile.xml you can see how it's all tied together.

    Regards,
    Mikael Svenson 


    Search Enthusiast - MCTS SharePoint/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    • Proposed as answer by Gunnar Braaten Friday, June 24, 2011 10:49 AM
    • Marked as answer by Ben Liang Friday, June 24, 2011 1:23 PM
    Friday, June 24, 2011 9:25 AM
  • Mikeal, 

    I'd glad you think the same that it does not make sense where Level 1 has weight higher than Level 2 and 3. The stop word threshold is other factor here that I didn't think of. Thanks for the nice write-up. 

    Ben

    Friday, June 24, 2011 1:26 PM