none
FAST ESP Did you mean suggestion RRS feed

  • Question

  • Hi All,

          While searching for "vistor", FAST ESP "Did You Mean" is suggesting "vis tor". But i like fast esp to display "visitor" instead of "vis tor" if "visitor" is available on dictionary & if "visitor" is not available in dictionary, it should not suggest anything rather than suggesting a word that is not in dictionary. What kind of change i have to do, to make this possible? I have checked few things in my side -

    1. FAST ESP is providing the suggestion on string based search on dictionary file. Like if I am having a word "visioview" in my dictionary file, query is taking "vis" from "visioview" and giving a output like that. Similarly it is giving output of "tor" from a word "torrent" from dictionary file.

    We have not implemented any phonetic concept. Hence if a word is not available on dictionary, should not be suggested. But here FAST ESP is suggesting something with string search of dictionary file. Again we have tried to compile dictionary with " " in each word. But working with this is bit difficult as it is providing search with EXACT MATCH ONLY even with case sensitive. Our dictionary looks as follows:

    John   3

    Mikel  10

    visioview 2

    torrent 1

    Wednesday, June 6, 2012 7:59 AM

All replies

  • You need to add visitor to spell check dictionary.


    Harvail

    Friday, June 8, 2012 8:42 AM
  • You need to add visitor to spell check dictionary.


    Harvail

    Harvail,

    I think you didn't get my issue... please check the line ... "Hence if a word is not available on dictionary, should not be suggested"

    Friday, June 8, 2012 9:05 AM
  • It is in dictionary, that is why you are getting suggestion. There is no other logic to display suggestions except dictionaries.

    generally FAST ESP dictionaries tool does allow you to see only custom entries.

     try to use in qr server interface,these options along with others options which you are using

    qtf_didyoumean:addconsidered=on

    qtf_didyoumean:consideredverbose=on

     it will tell you which dictionary is giving this suggestions.

    To me it looks is Phrase suggestion dictionary.


    Harvail

    Friday, June 8, 2012 9:33 AM
  • Hello Harvail,

    I am not saying that Did You Mean should suggest that is not present in the dictionary. Its showing something by WORD SPLITTING. I don't want "visitor" should show in suggestion. Please check this line "But i like fast esp to display "visitor" instead of "vis tor" if "visitor" is available on dictionary & if "visitor" is not available in dictionary, it should not suggest anything rather than suggesting a word that is not in dictionary." We want to stop this word splitting concept. "VISITOR" is not inside the dictionary, I have checked this. I want Did You Mean should not suggest anything as it is not present in dictionary. But in our case, Did You Mean is suggesting something else. Please read my 1st post carefully. In case of you need any other info from me... I am ready to provide you.

    Regards,

    Rituparna

    Friday, June 8, 2012 9:41 AM
  •  I am also strugling with a same kind of issue .. I am searching "Alesander" and want FAST to show up "Did you mean :Alexander" . Here I must tell that "Alexander" is in the dictionary. However FAST is suggesting me "Did you mean : ale sander" ... this ale sander is not in our dictionary ... Digging dip I found that it is searching the "ale" from Alexander and similar such entries where "ale" is present .... and similarly "Sander" from some other other similarly... Please help me on this as well.....Someone... 
    Friday, June 8, 2012 10:13 AM
  • I am saying all suggestion comes from dictionaries. This word is not present spellcheck dictionary but present in other dictionaries.

    You can use this parameter while submitting the query. You will not get the suggestion.

    qtf_didyoumean:rewrite_pre=off

    Further more you can also use

    qtf_didyoumean:phrasing=off


    Harvail

    Friday, June 8, 2012 10:33 AM
  • I agree with Harvail – in this situation, we would generally recommend either removing the two small terms from the spellcheck dictionary, or turn off word splitting at query time:

    qtf_didyoumean:rewrite_pre=value  (on,off,suggest)

    For example:

    http://<qrserver>:15100/cgi-bin/xsearch?offset=0&hits=10&query=vistor&spell=suggest

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY="" CUSTOM="vistor->vis tor" MESSAGE="Suggested rewrite for string" MESSAGEID="12" INSTANCE="didyoumean"/>

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY="vis tor" CUSTOM="vistor" MESSAGE="Spellcheck suggestion for string node" MESSAGEID="14" INSTANCE="didyoumean"/>

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY=""vis tor"" CUSTOM="" MESSAGE="Spellcheck suggestion for query" MESSAGEID="2" INSTANCE="didyoumean"/>

    http://<qrserver>:15100/cgi-bin/xsearch?offset=0&hits=10&query=vistor&spell=suggest&qtf_didyoumean:rewrite_pre=off

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY="" CUSTOM="vistor->visitor" MESSAGE="Spellcheck suggestion for term" MESSAGEID="4" INSTANCE="didyoumean"/>

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY="visitor" CUSTOM="vistor" MESSAGE="Spellcheck suggestion for string node" MESSAGEID="14" INSTANCE="didyoumean"/>

    <QUERYTRANSFORM NAME="FastQT_DidYouMean" ACTION="Suggested new query" QUERY=""visitor"" CUSTOM="" MESSAGE="Spellcheck suggestion for query" MESSAGEID="2" INSTANCE="didyoumean"/>

    Thanks!

    Rob Vazzana | Sr Support Escalation Engineer | US Customer Service & Support

    Customer Service   & Support                            Microsoft| Services

    • Proposed as answer by Dheeman Friday, June 8, 2012 7:16 PM
    Friday, June 8, 2012 1:26 PM
    Moderator
  • Hi Rituparna, you could allign the spell check dictionary as per the content that has been indexed.

    There are tow stages to it

    (i) Extracting the content that's been indexed (this can be accomplished with the help of the createspelldata command) and

    (ii) The actual allignment (accomplished with the help of the tunespellcheckfl command.

    Before running the createspelldata command we need to ensure that a significant volume of the content has already been indexed.

    You may also want to truncate the entries in the dictionary before issuing the createspelldata and the tunespellcheckfl command. That way we can be sure that the dictionary contains only those terms which are present in the index.

    Please let me know if you would like to have further assistance with respect to how the createspelldata and tunespellcheckfl commands are used....I believe with this you would be able to meet your business requirement i.e. if a term is indexed only then should a Did you mean suggestion be made, else no suggestions should be made.


    Suraj

    Friday, June 8, 2012 3:40 PM
  • Thanks Rob and Harvail for solving my earlier issue of word splitting .. But now I am facing a different issue.

    Suppose I search for Lamy Jakaria .... FAST suggests me "Did You Mean Lammy Zakaria" which is correct. But then I change my query to

    "Lamy Jakariaa" ... FAST suggests me "Did you mean " Lammy Zakariaa" .... this is not okay with me because there in no word in dictionary named Zakariaa.. instead it would be ok with me if it had given me as "no results found" if it could not suggest me the actual "Lammy Zakaria"...

    Please Help ... Someone...

    Thanks in advance...

    Friday, June 8, 2012 7:21 PM
  • Thanks  Harvail for solving my earlier issue of word splitting .. But now I am facing a different issue.

    Suppose I search for Lamy Jakaria .... FAST suggests me "Did You Mean Lammy Zakaria" which is correct. But then I change my query to

    "Lamy Jakariaa" ... FAST suggests me "Did you mean " Lammy Zakariaa" .... this is not okay with me because there in no word in dictionary named Zakariaa.. instead it would be ok with me if it had given me as "no results found" if it could not suggest me the actual "Lammy Zakaria"...

    Please Help ... Someone...

    Thanks in advance...
    Friday, June 8, 2012 7:21 PM
  • Hello,

    This is a different issue, and I suspect this is an issue of propername/phrase.  I imagine that you have “Lammy” as an entry in the propername dictionary already.  Without seeing the dictionaries, I believe all that you have to do is add an entry for the first and last name, “Lammy Zakaria”.

    Thanks!

    Rob Vazzana | Sr Support Escalation Engineer | US Customer Service & Support

    Customer Service   & Support                            Microsoft| Services

    Friday, June 8, 2012 7:49 PM
    Moderator
  • Thanks Rob for the quick reply...

    I have like  this in my dictionary

    Lammy  3

    Zakaria  1

    .....

    Where 3 and 1 are the number of times Lammy & Zakaria occurs in the total list of people names  . This is a new dictionary I created for all names (firstname and lastname) that exist , using dictman ..

    Now do you mean to say apart from the above entries, do I need to enter the full name as well ??

    Lammy Zakaria   1

    Please help......

    Saturday, June 9, 2012 6:25 AM
  •  Dheeman,

       By Default ESP gives suggestion by changing position of (_adding/removing) one char only. In case of "Lamy Zakariaa" , it applied the suggestion on Lamy--->Lammy and ingnored the second word.

       Try to "Lammy Zakariaa", it should suggest Zakariaa--> Zakiara.

       Let me know, if it helps.


    Harvail

    Monday, June 11, 2012 9:17 AM
  • Thanks Harvail...

    So if I want to increase dependedncy from one char to three chars ..what should I do ?

    Monday, June 11, 2012 12:10 PM
  •  Dheeman,

       By Default ESP gives suggestion by changing position of (_adding/removing) one char only. In case of "Lamy Zakariaa" , it applied the suggestion on Lamy--->Lammy and ingnored the second word.

       Try to "Lammy Zakariaa", it should suggest Zakariaa--> Zakiara.

       Let me know, if it helps.


    Harvail

    Hi Harvail,

    We are also facing the same kind of issue and we want to Increase this dependency so that FAST should also include the second character also as you mentioned. May be this can be done by modifying XML. But I am unable to find it out exactly which XML file do I need to change. Kindly let me know the procedure.

    Rituparna

    Wednesday, June 13, 2012 6:54 AM
  • Rob , by default FAST Did You Mean can suggest 1 correction. I would need to increase this to 3 . Can you please help me , on how do I aceive this and and if there are any associated steps in doing this.?

    Thanks

    Wednesday, June 13, 2012 9:11 AM
  • I am not sure  how it is going to work, I never tried following setting.

    This is what Advance Linguistic Guide says. 

    $FASTSEARCH/etc/config_data/QRServer/webcluster/etc/qrserver/didyoumean/spellcheck.<lang>.xml 

    tolerance integer 2

    The similarity of two terms to each other can be measured in terms of the number of edit operations (deletion,substitution, transposition, addition) that are necessary to get from one term to the other. For example, one edit operation (deletion) is necessary to get from window to window. The parameter tolerance determines the number of edit operations that are allowed to get from one term to the other.


    Harvail

    Wednesday, June 13, 2012 9:18 AM
  • Thanks Harvail.. I saw this as well , but I am confused a bit becasue if default value is 2 , how it is that FAST ESP suggests 1 edit by default... So in order to allow 3 edits do I need to put a value of 3 or 4..

    Thanks

    Wednesday, June 13, 2012 9:26 AM
  • I am not sure  how it is going to work, I never tried following setting.

    This is what Advance Linguistic Guide says. 

    $FASTSEARCH/etc/config_data/QRServer/webcluster/etc/qrserver/didyoumean/spellcheck.<lang>.xml 

    tolerance integer 2

    The similarity of two terms to each other can be measured in terms of the number of edit operations (deletion,substitution, transposition, addition) that are necessary to get from one term to the other. For example, one edit operation (deletion) is necessary to get from window to window. The parameter tolerance determines the number of edit operations that are allowed to get from one term to the other.


    Harvail

    I tired to change the tolerance value , restarted the QRServer(s) .. it did not work... Any ideas why ?

    Thanks

    Sunday, June 17, 2012 4:29 PM