none
Search crawl slow for one Content source RRS feed

  • Question

  • Hello,

    Since a few weeks I'm having issue with one of my Content Source to crawl, it's very slow. In my search service, I have 3 Content source, 2 of them run really quickly but the other is REALLY slow. Last week the incremental was still running for 110 hours, so I decided to stop the incremental crawl on that content source then start a new FULL. It took 56 hours, which I think is normal for this environment and at least during the FULL crawl, the crawl rate was at around 38 documents / second.

    After that FULL was completed, the incremental started, now it's been running for 17 hours and still not completed. The crawl rate is 0.03 items / second. If I look at the CRAWL LOG for this specific content source, I see that sometime, no files get crawled for a few minutes which is why this is really slow.

    My other content sources are still running fine and fast.

    I've read a few blogs about doing an Index reset, but I don't want to do that. What could explain the fact that on the same Search service, some content source is running fast and one is not?

    thank you

    Friday, April 27, 2018 3:25 PM

All replies

  • Hi,

    Does crawling this content source cause high CPU and Memory?

    It seems that the issue is caused by specific web application or site collection, not search service application.

    You need to look in the ULS logs to figure out the root cause.

    A demo for your reference:

    https://stuffandtacos.azurewebsites.net/2015/11/05/sharepoint-slow-search-crawling-rate-and-low-document-per-seconds-indicator/

    Best Regards,

    Dean Wang


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Tuesday, May 1, 2018 8:03 AM
    Moderator
  • Hello,

    No I don't see any high CPU/memory usage on my Search servers. What I found, is during the incremental if I go to the Crawl log of that Content source, looks like it's stuck for a long time on a site collection that has a lot of SharePoint Groups (around 3000).

    I'm trying to find a way in the ULS logs to confirm that this might slow the crawl...

    I've seen the post you provided, I was also in the process of asking my DBA to try it, but the search DB are kind of really small, so I was having some doubts.

    Tuesday, May 1, 2018 12:28 PM
  • Hello,

    After some troubleshooting, I can tell that during the incremental crawls, it gets really slow on all sites that are hosted on the same Content database...

    See everything was fast from 5/5/2018 12:14 PM then nothing happened 5/6/2018 10:08PM, that's almost 18 hours of idle... And now every crawl are slow. I'm not show the site collections on that screenshot, but all the other sites/items, are all in the same content database.

    Monday, May 7, 2018 1:19 PM