locked
Optimized Index Server Infrastructure RRS feed

  • Question

  • Hi,

    I have 4 front end server. index & query is installed in app server.Two database server.25 live sites are running. Every site has it own content DB, Web Application & application pool.Many batch jobs is running in index server to push content sites. Overall data volume more than 100GB. problem is indexing is not working sometimes. Sometimes incremental crawling is hanged.Sometimes it is running too long. So what should be the optimized server infrastructure by which I can solve this problem. Please help.I am using MOSS 2007. My product catalog is dependant on managed proerty so if crawl fails then the whole thing are messed up.

    Please let me know if you have any query.

     


    thanksArup MCTS - SharePoint

    Play Sudoku Online
    • Edited by Arup Biswas Thursday, December 23, 2010 7:44 PM
    Thursday, December 23, 2010 9:22 AM

Answers

  • Do you run Query role on WFE servers and Index server? If yes this configuration is not supported by SharePoint. You have to run Query role on WFE servers and Index role on Application server or Query and Index just on Application. Read this http://technet.microsoft.com/en-us/library/cc263044(office.12).aspx


    Oleg
    • Marked as answer by Peng Lei Monday, December 27, 2010 7:29 AM
    Thursday, December 23, 2010 4:26 PM
  • If your Crawl is hanging or taking a long time then I would suspect the problem may be the structure of the content, memory or perhaps a bad iFilter.

    The incremental crawl I believe runs a off a change log in the content DB so if you have batch jobs which change a lot of content then the incremental crawl will be prolonged, perhaps you can optimize the way that you push content to be more discreet?

    A common mistake if you are using PDFs is to install the 32bit PDF iFilter which is actually Acrobat 6.  For PDF on 32bit servers ensure that you are using the iFilter in the 9 reader or better still foxit (faster).

    Look at your crawl logs and see if there is any thing in there in terms of errors.

    If you have folders containing lots of items (i.e. big document libraries or lists) then the crawler will load the entire contents of the list into memory before commencing the crawl and can run out of memory.  There are registry settings to increase the amount of memory available to the crawler.  64bit will make life a lot easier here.

    If it helps I crawled 1Tb of data on a 32bit sharepoint farm with 3 web front servers (query role), 1 index server and crawl on all WFEs enabled.  It took a long time but it worked.  What also did help was setting up a content source per site collection in case a crawl did crash a full crawl was only required on a single site collection and not the whole thing.

     

    • Marked as answer by Peng Lei Monday, December 27, 2010 7:29 AM
    Thursday, December 23, 2010 6:30 PM

All replies

  • Do you run Query role on WFE servers and Index server? If yes this configuration is not supported by SharePoint. You have to run Query role on WFE servers and Index role on Application server or Query and Index just on Application. Read this http://technet.microsoft.com/en-us/library/cc263044(office.12).aspx


    Oleg
    • Marked as answer by Peng Lei Monday, December 27, 2010 7:29 AM
    Thursday, December 23, 2010 4:26 PM
  • If your Crawl is hanging or taking a long time then I would suspect the problem may be the structure of the content, memory or perhaps a bad iFilter.

    The incremental crawl I believe runs a off a change log in the content DB so if you have batch jobs which change a lot of content then the incremental crawl will be prolonged, perhaps you can optimize the way that you push content to be more discreet?

    A common mistake if you are using PDFs is to install the 32bit PDF iFilter which is actually Acrobat 6.  For PDF on 32bit servers ensure that you are using the iFilter in the 9 reader or better still foxit (faster).

    Look at your crawl logs and see if there is any thing in there in terms of errors.

    If you have folders containing lots of items (i.e. big document libraries or lists) then the crawler will load the entire contents of the list into memory before commencing the crawl and can run out of memory.  There are registry settings to increase the amount of memory available to the crawler.  64bit will make life a lot easier here.

    If it helps I crawled 1Tb of data on a 32bit sharepoint farm with 3 web front servers (query role), 1 index server and crawl on all WFEs enabled.  It took a long time but it worked.  What also did help was setting up a content source per site collection in case a crawl did crash a full crawl was only required on a single site collection and not the whole thing.

     

    • Marked as answer by Peng Lei Monday, December 27, 2010 7:29 AM
    Thursday, December 23, 2010 6:30 PM