locked
ContentIndexState : FailedAndSuspended RRS feed

  • Question

  • Dear readers,

    I have a problem on a Exchange 2016 server.

    This started when I updated  the Exchange server from CU6 to CU16.

    The Exchange Search database is not working anymore.

    I have tried the fallowing:

    - Stopping the Exchange Search Services and deleted (backup) the mailbox guid directory. (guid.single)

    Result, It starts with crawling and ends up with FailedAndSuspended.

    - Reinstalled the Search host-controller .

    https://www.experts-exchange.com/questions/29172216/CONTENT-INDEX-STATE-FOR-ALL-DATABASES-APPEAR-AS-UNKNOWN.html

    (See answer Saif)

    Same result, it starts with Crawling and ends up with FailedAndSuspended.

    - Updated latest Windows update's and restarting the machine.

    The mailbox database is 206 GB

    The Exchange Server has 13 GB Ram (virual machine / vmware) - Planning to upgrade this soon.

    The Exchange disk has more then 120% free space.

    The biggest mailbox is 123 GB (company mail info mailbox)  <-currently trying to make this smaller / archiving.

    We also have a 2nd Database on the same server (called Archive) - (11 GB in size)

    This database has the same indexing error: FailedAndSuspended. (also cleared guid.single folder)

    Advise would be welcome.

    Friendly regards,

    Pascal Kroeze

    Microsoft System Engineer


    Friday, May 1, 2020 12:01 PM

All replies

  • Hi Pascal,

    Kindly check the Windows application events logs and share the Error & warning for check this isssue.

    Friday, May 1, 2020 12:55 PM
  • The indexing of mailbox database Mailbox Database 1759433534 encountered an unexpected exception. Error details: Microsoft.Exchange.Search.Core.Abstraction.OperationFailedException: The component operation has failed. ---> Microsoft.Exchange.Search.Engine.FeedingSkippedForCorruptionException: "Feeding was skipped for '48f0b7fa-ad8f-4e76-b10d-7f5f16024f6d (Mailbox Database 1759433534)' due to the state 'Failed', error code: 'CatalogCorruption', failure code: '4321345', failure reason: 'Journal has been detected as invalid.'."
       at Microsoft.Exchange.Search.Engine.SearchFeedingController.InternalExecutionStart()
       at Microsoft.Exchange.Search.Core.Common.Executable.InternalExecutionStart(Object state)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Search.Core.Common.Executable.EndExecute(IAsyncResult asyncResult)
       at Microsoft.Exchange.Search.Engine.SyncRootController.ExecuteComplete(IAsyncResult asyncResult)


    The indexing of mailbox database Archive encountered an unexpected exception. Error details: Microsoft.Exchange.Search.Core.Abstraction.OperationFailedException: The component operation has failed. ---> Microsoft.Exchange.Search.Engine.FeedingSkippedForCorruptionException: "Feeding was skipped for 'f43b8d9d-7ae9-4bb8-8c82-577aa44bac09 (Archive)' due to the state 'Failed', error code: 'CatalogCorruption', failure code: '4321345', failure reason: 'Journal has been detected as invalid.'."
       at Microsoft.Exchange.Search.Engine.SearchFeedingController.InternalExecutionStart()
       at Microsoft.Exchange.Search.Core.Common.Executable.InternalExecutionStart(Object state)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Search.Core.Common.Executable.EndExecute(IAsyncResult asyncResult)
       at Microsoft.Exchange.Search.Engine.SyncRootController.ExecuteComplete(IAsyncResult asyncResult)

    Another error:

    The FastFeeder component received a connection exception from FAST. Error details: Microsoft.Exchange.Search.Fast.FastConnectionException: Connection to the Content Submission Service has failed. ---> Microsoft.Ceres.External.ContentApi.ConnectionException: Recovery failed after 0 retries
       at Microsoft.Ceres.External.ContentApi.DocumentFeeder.DocumentFeeder.CheckRecoveryFailed()
       at Microsoft.Ceres.External.ContentApi.DocumentFeeder.DocumentFeeder.SubmitDocument(Document document, TimeSpan timeout)
       at Microsoft.Ceres.External.ContentApi.DocumentFeeder.DocumentFeeder.SubmitDocument(Document document)
       at Microsoft.Exchange.Search.Fast.FastFeeder.SubmitDocumentInternal(Object state)
       --- End of inner exception stack trace ---


    When I check the process it seems its bouncing between healthy and upgrading, crawling, failed, failed and suspended, back to failed, crawling, etc etc.

    In perfmon you can see some changes, but mailboxes remaining are still max and not counting down.

    Items sent for processing and rate of attempted items Submission are both counting and then reset back to 0 after a time.

    It's seems the system the process keeps failing and stuck in a loop.

    It's also draining the server resources, the server is very slow.

    Update: It seems the loop is over:

    Final state: FialedAndSuspended

    Perfmon crawling monitoring is all set to 0.

    Friday, May 1, 2020 5:20 PM
  • Hi Pascal,

    Since this issue starts after upgrading to Exchange 2016 CU16, please make sure .NET Framework 4.8 and Visual C++ Redistributable Packages for Visual Studio 2013 are installed for CU16.

    Please use the Test-ExchangeSearch cmdlet for the database for a specific mailbox:

    Test-ExchangeSearch -MailboxDatabase <db name> -Verbose

    I noticed that you are going to upgrade the Exchange RAM. It's suggested to reseed the content index catalog after upgrading RAM, to make sure this issue is not caused by the Exchange performance. For your reference: Reseed the content index catalog if there is only one copy of the mailbox database.

    If the issue persists, try to recreate monitoring mailboxes. You can check the detailed steps from this blog: Exchange 2013/2016 Monitoring Mailboxes.

    Regards,

    Lydia Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Monday, May 4, 2020 6:16 AM
  • Just checking in to see if above information was helpful. If you would like further assistance, please feel free to reply this post directly so we will be notified to follow it up.

    Regards,

    Lydia Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Friday, May 8, 2020 8:48 AM
  • Here is a brief summary for replies above. Hope more people can get useful information from it.

    Issue Symptom:

    I have a problem on a Exchange 2016 server.

    This started when I updated  the Exchange server from CU6 to CU16.

    The Exchange Search database is not working anymore. This database has the same indexing error: FailedAndSuspended. When I check the process it seems its bouncing between healthy and upgrading, crawling, failed, failed and suspended, back to failed, crawling.

    Possible Cause:

    Some components don't work well after the upgrade.

    Troubleshooting Steps so far:

    Check the Windows application events logs.

    Since this issue starts after upgrading to Exchange 2016 CU16, please make sure .NET Framework 4.8 and Visual C++ Redistributable Packages for Visual Studio 2013 are installed for CU16.

    Please use the Test-ExchangeSearch cmdlet for the database for a specific mailbox:

    Test-ExchangeSearch -MailboxDatabase <db name> -Verbose

    Next Step:

    It's suggested to reseed the content index catalog after upgrading RAM, to make sure this issue is not caused by the Exchange performance. 

    If the issue persists, try to recreate monitoring mailboxes. 

    Reference Links:

    Reseed the content index catalog if there is only one copy of the mailbox database

    Exchange 2013/2016 Monitoring Mailboxes

    Regards,

    Lydia Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Thursday, May 14, 2020 8:59 AM