none
How can I debug the index data?

    Question

  • Hi All,

    In SharePoint 2013, I just wonder that how can I debug the index data from content processing process?

    Let's say with FAST for SharePoint, I can look into the FIXML data to verify the content which is very useful.

    So, I wonder that is there a way to verify the content in index or the content from the output of content processing process?

    Best Regards,

    Andy

    Wednesday, October 31, 2012 7:20 AM

Answers

  • In SP 2013 search, the index stored on disk in binary format within the IndexComponent/storage/data folder. (There is a folder named with the GUID of the Search Service ID).

    Currently we do not have an external tool for viewing documents inside the index.  We are working with the PG on this.

    No indexed data is stored in the database. (Only admin and config data is stored there)

    Also... no more FIXML.

    There is no longer a need for FIXML due to:

    - the new document model (created by the new content processing engine) which is sent directly to an in-memory journal, then to the indexer. (we currently do not have the ability to look into the journal)

    - the new partial update feature which essentially divides the index in to update groups containing different sets of managed properties/field. This allows the indexer to update only a section of the indexed document as opposed to entire document.

    Ex: Updating all document ACL info is now a light(er) weight operation because only a section of the index is getting reprocessed.

    Hope this helps!

    Tuesday, November 06, 2012 7:27 PM
  • According to internal trainning material:

    A new mechanism called 'partial update' replaces the need for FIXML. Updating a few managed properties (fields) of a document is now a light-weight operation.

    Previously, when performing a partial update, the indexer would have to read the whole FIXML document, find the element (Ex: ACL value), update the file, save it, then re-index the FIXML document.

    The main advantage of this is:

    - Quicker partial update process

    - Search system consumes less disk space

    SharePoint 2013 search repository in my system can be found in address like C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications\Search\Nodes\EE45BF\IndexComponent1\storage\data\SP7a055532d3c5.1.I.0.0

    Tuesday, November 06, 2012 7:17 AM
    Moderator
  • Hi, in order to understand & debug search you might want to review ExplainRank page functionality.
    Friday, January 25, 2013 12:34 PM

All replies

  • According to internal trainning material:

    A new mechanism called 'partial update' replaces the need for FIXML. Updating a few managed properties (fields) of a document is now a light-weight operation.

    Previously, when performing a partial update, the indexer would have to read the whole FIXML document, find the element (Ex: ACL value), update the file, save it, then re-index the FIXML document.

    The main advantage of this is:

    - Quicker partial update process

    - Search system consumes less disk space

    SharePoint 2013 search repository in my system can be found in address like C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications\Search\Nodes\EE45BF\IndexComponent1\storage\data\SP7a055532d3c5.1.I.0.0

    Tuesday, November 06, 2012 7:17 AM
    Moderator
  • Thanks Gu,

    Anyway, I am quite confuse that where is the index gonna be stored? Is it in SQL Server or the SharePoint Server?

    Regarding FAST for SharePoint 2010, the actual index is in file structure in the FAST machine.

    I need to confirm this because it does impact the sizing of the storage space on MSSQL/SharePoint.

    Thanks and Regards,

    Andy

    Tuesday, November 06, 2012 7:33 AM
  • In SP 2013 search, the index stored on disk in binary format within the IndexComponent/storage/data folder. (There is a folder named with the GUID of the Search Service ID).

    Currently we do not have an external tool for viewing documents inside the index.  We are working with the PG on this.

    No indexed data is stored in the database. (Only admin and config data is stored there)

    Also... no more FIXML.

    There is no longer a need for FIXML due to:

    - the new document model (created by the new content processing engine) which is sent directly to an in-memory journal, then to the indexer. (we currently do not have the ability to look into the journal)

    - the new partial update feature which essentially divides the index in to update groups containing different sets of managed properties/field. This allows the indexer to update only a section of the indexed document as opposed to entire document.

    Ex: Updating all document ACL info is now a light(er) weight operation because only a section of the index is getting reprocessed.

    Hope this helps!

    Tuesday, November 06, 2012 7:27 PM
  • Hi Jeremy!

    Any news about "external tool for viewing documents inside the index". Would be very handy indeed to understand&debug search!

    br,

    -hk

    Monday, January 21, 2013 11:00 AM
  • Hi, in order to understand & debug search you might want to review ExplainRank page functionality.
    Friday, January 25, 2013 12:34 PM
  • Hi GuYuming, where can i get the Training Metrial and Technical Referance document to implement "partical update" and custom security trimming in SharePoint 2013/2016 Search?
    Friday, November 13, 2015 11:37 AM