none
Incremental crawling - Updates not showing up RRS feed

  • Question

  • I am using the JDBC connector to index a table (clients).  i would like to setup a second connector to index a second table (Phone) that is related to client by ClientId key.  I made a second JDBC connector and set it to Update documents rather than add them.  the connector runs and states many docs were updated but i do not see any changes when i search again.

    Does anybody know what I can do to setup multiple connectors to each partially update a document?  I need this for 15-20 secondary tables, but cannot even get 1 working at the moment.

     

    Thanks.

    Wednesday, August 3, 2011 11:32 PM

All replies

  • Hi

    It would be much easier if you could combine all these tables into a single view, so that you crawl all the information in one go. Is that not possible in your scenario? If a normal view does not provide you enough flexibility, I think you can even crawl the outputs of a stored procedure, in which you can do all sorts of complex joins, lookups, etc.

     


    Thomas Svensen | Microsoft Enterprise Search Practice
    Friday, August 5, 2011 11:04 AM
    Moderator
  • Thomas,

    Thanks for the reply.  I have a few hundred million records in my main table and between 1-6 times that many records in each of the secondary joined tables.  I feel a view would be too slow to use on this much data.  Any suggestions on crawling all this content efficiently?

     

    I was hoping to be able to crawl each table independently and have FAST be able to update the appropriate document with each result set (all joined by client id).

    Friday, August 5, 2011 6:44 PM
  • Thomas,

    Another thing I forgot to mention : I am making use of the JDBC Connector group 'Output', specifically the OperationMode paramter.  This is what I set to 'update' rather than 'add', but this config option is not documented on MSDN.  Any info on this and if it is what I should use for partial document updates?

     

    Thanks.

    Friday, August 5, 2011 9:21 PM
  • Hi again

    I am sorry, but I do not have too much hands-on experience with using the JDBC Connector, so I cannot suggest best-practices regarding those config parameters. I was not really aware of the fact that you could do partial updates via this tool, and I think very few people have used this feature.

    But I would like to go back to your database model. Given that all these tables are indexed properly, I don't see why it should be a problem to join them and feed them all at once. It has these advantages:

    1. Better performance, since indexing is just done once for each item. Partial updates are actually very costly, since it's effectively a full reindexing.
    2. Simpler, more standardized model - easier for others to understand, and a better supported scenario
    3. Allows switching to BCS, which is the recommended interface, and also the one most likely to be supported in "Office 15"
    4. Tuning and optimization can be done on the database side, which is generally more transparent and well-documented than on the indexing side

    Just my 2 cents.

    Regards


    Thomas Svensen | Microsoft Enterprise Search Practice
    Wednesday, August 10, 2011 6:17 AM
    Moderator