none
CustomerExtensibility stage not hit on incremental security-only crawls? RRS feed

  • Question

  • Hi everyone,

    I have a pipeline extensibility stage that adds some metadata to documents going through the pipeline. For full crawls this works as expected. For debugging purposes I employed two Spy stages. One right before and the other right after the CustomerExtensibility stage. This shows me that the set of document metadata is as expected. However if I change only permissions on an item in SharePoint and start an incremental crawl, no metadata is added as this item goes through the pipeline. Spy-Before output and Spy-After output do not differ. I have a slight suspicion that the CustomerExtensibility stage does nothing on security-only crawls. Is this true or might there be a problem with my setup?

    Thanks in advance!

    Michael

    Thursday, April 5, 2012 6:37 AM

Answers

  • Hi,

    CustomerExtensibility stages are run on new or modified content, that is on the "ADD" operation in FAST. A security crawl will do a "PARTIALUPDATE" as only meta data related to ACL's has changed, and thus it will skip most pipeline stages, including your custom ones. This yields quicker indexing for security only changes.

    Do you have a reason to run your custom code when only ACL's have changed?

    Regards,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/

    • Proposed as answer by SandipSingh Wednesday, April 11, 2012 1:41 PM
    • Unproposed as answer by SandipSingh Wednesday, April 11, 2012 1:41 PM
    • Marked as answer by Michael Sliwak Thursday, April 12, 2012 7:51 AM
    Wednesday, April 11, 2012 10:59 AM

All replies

  • Hi,

    CustomerExtensibility stages are run on new or modified content, that is on the "ADD" operation in FAST. A security crawl will do a "PARTIALUPDATE" as only meta data related to ACL's has changed, and thus it will skip most pipeline stages, including your custom ones. This yields quicker indexing for security only changes.

    Do you have a reason to run your custom code when only ACL's have changed?

    Regards,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/

    • Proposed as answer by SandipSingh Wednesday, April 11, 2012 1:41 PM
    • Unproposed as answer by SandipSingh Wednesday, April 11, 2012 1:41 PM
    • Marked as answer by Michael Sliwak Thursday, April 12, 2012 7:51 AM
    Wednesday, April 11, 2012 10:59 AM
  • Hi Mikael,

    thanks for your response. My intention is to parse the security descriptor in docaclms and add some additional metadata depending on the parse result. I know that this is rather unsupported, so I have to evaluate it once again. :-)

    Thanks!

    Thursday, April 12, 2012 7:50 AM
  • Hi,

    You can update C:\FASTSearch\etc\processors\customerextensibility.xml and change

    <ops><add/></ops>

    to

    <ops><add/><partialupdate/></ops>

    That will make your code run on partial updates as well. Of course, this is unsupported :)

    Thanks,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/

    Thursday, April 12, 2012 9:10 AM
  • Hi Mikael,

    Yep, this also came to my mind. Like in the old ESP days. ;-)

    Thanks again!

    Thursday, April 12, 2012 10:13 AM
  • To revive this thread again.

    I have enabled the CustomerExtensibility stage for partial updates. Now everything works fine in the pipeline. My additional metadata gets also added to the document on partial updates. I verified this by employing a Spy stage at the end of the pipeline. However the change property does not get reflected in the Fixml. It still contains the old value.

    Any ideas on this?

    Thanks,
    Michael

    Thursday, May 24, 2012 2:09 PM
  • Hi,

    You probably have to add partialupdate support to more stages as well in order to reprocess the values into the fixml. I'll leave it to you to experiment with it :)

    Thanks,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/MCT
    http://techmikael.blogspot.com/
    Author of Working with FAST Search Server 2010 for SharePoint

    Thursday, May 24, 2012 2:52 PM