Pipeline Extensibily for getting data from DB RRS feed

  • Question

  • Hi,

    I have some documents in fileshares and i have the metadata properties associated with these in a DB.I 'm planning a custom pipeline stage which passes the document url to the db and retrieve the associated metadata properties and map them to some new crawled properties.Is this solution a viable one?

    Wednesday, July 6, 2011 2:04 PM

All replies

  • Hi Rush-87

    I wouldn't recommend this approach for a couple of reasons:

     1. If you make changes to the metadata only, an incremental crawl of the file share will not pick it up

     2. A database lookup for every request will be quite costly, as you will not use any DB connection pooling or similar

    But with those limitations in mind, it should still work, especially for a small set of documents, where you can do regular full crawls.

    A better approach, albeit requiring some more work, is to write a BCS connector which reads from the database and then pulls in file from the file share and passes it on as a blob. I have some internal material on the required steps. If you are interesting, I'll see if I can something publicly available.


    Thomas Svensen | Microsoft Enterprise Search Practice
    Thursday, July 7, 2011 9:53 PM
  • Hi Rush,

    it could be a viable option but you would need to implement some sort of pipeline framework where you can cache all the DB values to prevent one DB call per row.

    So I'm with Thomas on this one :) Go BCS.

    Mikael Svenson 

    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    Sunday, July 10, 2011 9:06 PM
  • Hi Mikael and Thomas,

    Thanks for your advice... Il go with u guys in this..



    Monday, July 11, 2011 11:46 AM