none
Special Character in Refinement

    Question

  • Using FQL, how can I filter the search results by a managed property which contains special character, such as - ?

    The problem I have is: I am using Refinement filter like Attribute:equals(1-2) to filter result items. But zero results will return.

    If I change the refinement filter to Attribute:phrase(1-2), I will return the results I need. However, phrase will also return results which Attribute has 1-2 m. This is as accurate as I want.

    It doesn't help if I add double quote on 1-2 or add backslash. Anybody have experience in handling special character in SP2013 search or FAST search?

    Monday, February 17, 2014 1:27 PM

All replies

  • Hi Arfanti,

    Based on the following article it is by design, the special character "-" will be treated as white space inside a string expression, this is related to the tokenization process.

    http://msdn.microsoft.com/en-us/library/office/ff394606.aspx

    Handling strings with special characters

    Special characters such as comma (","), semicolon (";"), colon (":"), period ("."), minus ("-"), underline ("_"), or forward slash ("/") are treated as white space inside a string expression enclosed in double quotation marks. This is related to the tokenization process. These characters also imply an implicit phrasing of the tokens separated by these characters.

    The following query expressions are equivalent.

    title:string("animals birds", mode="phrase")title:"animals/birds"title:string("animals/birds", mode="and")title:string("animals/birds", mode="or")

    The following query expressions are equivalent.

    title:or(string("animals birds", mode="phrase"), string("animals insects", mode="phrase"))title:string("animals/birds animals/insects", mode="or")

    The following query expressions are equivalent.

    body:string("help contoso com", mode="phrase")body:string("help@contoso.com")

    Tokenized phrase matching

    You can search for an exact string of tokens by using the STRING operator with mode="phrase" or the PHRASE operator.

    All such phrase operations imply a tokenized phrase match. This means that special characters such as comma (","), semicolon (";"), colon (":"), underscore ("_"), minus ("-"), or forward slash ("/") are treated as white space. This is related to the tokenization process.

    Thanks,
    Daniel Yang
    Forum Support
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Daniel Yang
    TechNet Community Support

    Tuesday, February 18, 2014 1:37 PM
    Moderator
  • I already read this MSDN article about FQL. The problem is about the hyphen symbol - in the refinementfilter.

    Example:

    http://www.contoso.com/sites/search/_api/search/query?querytext='*'&rowlimit=10&selectproperties='Title'&refinementfilters='Attribute:equals("1-2")'

    I can't get any results while the index does contain items which have 1-2 as value in Attribute managed property.

    Wednesday, February 19, 2014 1:01 PM
  • Hi Arfanti,

    I tried the SharePoint 2013 search query tool with your example query, it will return the results for the managed property (i.e. mptest in my test) with using refinementfilters='mptest:phrase(1-2)' like the following image.

    http://sp2013searchtool.codeplex.com/

    Thanks


    Daniel Yang
    TechNet Community Support

    Thursday, February 20, 2014 10:13 AM
    Moderator
  • Phrase doesn't do an exact match.

    In your Refinement Test list, if you set 1-2 22 in TestField for ITEM a, you will not only get the items which have 1-2 inTestField but will get this ITEM a also when setting mptest:phrase(1-2) in refinementfilter.

    Thursday, February 20, 2014 12:06 PM
  • What can we use if we want exact match & also special characters to be ignored
    Wednesday, June 04, 2014 7:20 PM
  • I agree this is very unintuitive behavior. Has anyone found good documentation on this. It seems odd that a full-text search for "1-2" would work and mprop:equals("1-2") would not yield anything.

    -DdJ

    Sunday, September 14, 2014 2:20 AM