none
Exchange Server 2019 CU3 - Time out for test thread RRS feed

  • Question

  • Hi guys,

    The Exchange Server 2019 CU3 is with problem on the Search Index. The users can't searching in OWA or Outlook app.

    The error is "Time out for test thread".

    The logs in event viewer is:

    An operation attempted against a FAST endpoint exprienced an exception. This operation may be retried. Error details: Microsoft.Exchange.Search.Fast.PerformingFastOperationException: An Exception was received during a FAST operation. ---> System.ServiceModel.FaultException: error CS1548: Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' -- 'Error signing assembly -- Unknown error (8013141c)'


    Server stack trace: 
       at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]: 
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at Microsoft.Ceres.ContentEngine.Admin.FlowService.IFlowServiceManagementAgent.PutFlow(String name, String serializedFlow)
       at Microsoft.Exchange.Search.Fast.FastManagementClient.AgentHandle`1.<>c__DisplayClass6_0.<PerformOperation>b__0(TManagementAgent agent)
       at Microsoft.Exchange.Search.Fast.FastManagementClient.PerformFastOperation[TManagementAgent,TResult](AgentHandle`1 agentHandle, Func`2 function, String eventLogKey)
       --- End of inner exception stack trace ---



    Tuesday, November 12, 2019 7:41 PM

Answers

  • AHHH finaly.

    The CU3 make anything stranger...

    I started doing some digging that pointed me at C:\Users\All Users\Microsoft\Crypto\RSA.

    I renamed this folder...

    AND run this:

    Invoke-Command -ComputerName SRVEX001 {Stop-Service MSExchangeFastSearch; Stop-Service HostControllerService}

    AND this

    Invoke-Command -ComputerName SRVEX001 {Start-Service MSExchangeFastSearch; Start-Service HostControllerService}

    Now the search service is ok.

    Very stranger the CU3 to do this...

    Tuesday, November 12, 2019 10:49 PM
  • Hi EvandroMalmsteen,

    It's great that you have solved the search issue, and thanks for your sharing. You can mark your reply as answer, his may help more people with similar problems. Here is a brief summary about this issue.

    Issue Symptom

    ====================================

    The Exchange Server 2019 CU3 is with problem on the Search Index. The users can't searching in OWA or Outlook app. The error is "Time out for test thread".

    The logs in event viewer is:

    An operation attempted against a FAST endpoint exprienced an exception. This operation may be retried. Error details: Microsoft.Exchange.Search.Fast.PerformingFastOperationException: An Exception was received during a FAST operation. ---> System.ServiceModel.FaultException: error CS1548: Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' -- 'Error signing assembly -- Unknown error (8013141c)'

    Possible Cause

    ====================================

    From the error information "Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' ", this issue may have relation with cryptographic key.

    Solution Summary

    ====================================

    Rename the folder MachineKeys under C:\Users\All Users\Microsoft\Crypto\RSA. Then restart Microsoft Exchange Search and Microsoft Exchange Search Host Controller services.

    Invoke-Command -ComputerName <name> {Stop-Service MSExchangeFastSearch; Stop-Service HostControllerService} Invoke-Command -ComputerName <name> {Start-Service MSExchangeFastSearch; Start-Service HostControllerService}

    Regards,

    Lydia Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Wednesday, November 13, 2019 6:42 AM
    Moderator

All replies

  • I tryed this, but not success.

    Issue Resolution

    To resolve the issue the following steps were taken in order.

    1. Stop the Microsoft Exchange Search and Microsoft Exchange Search Host Controller service.
    Invoke-Command -ComputerName SRVEX001 {Stop-Service MSExchangeFastSearch; Stop-Service HostControllerService}
     
    2. Remove all the files under “Fsis” file: 

    C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis
    We moved these files to another folder for backup purposes.

    3. Open EMS and “Run as Administrator"

    4. Navigate to directory C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Installer
    5. Launch the script and run the command below:

    .\installconfig.ps1 -action I -datafolder "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"
    This script will recreate the files under "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis" that we backed up/removed previously.

    6. Once the script installs the data, you need to uninstall and reinstall again (don't ask, it did not work for me without doing this):

    .\installconfig.ps1 -action U -datafolder "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"

    Then again installed Search component with below command :

    .\installconfig.ps1 -action I -datafolder "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"

    7. Force Active Directory Replication from a domain controller in the same AD Site as the Exchange 2019 server with repadmin /syncall /APeD.

    8. Check on the Exchange Server with Get-MailboxDatabaseCopyStatus and review the ContentIndexState.



    Tuesday, November 12, 2019 7:42 PM
  • About this error bellow, i added the iis users< IUSR_SERVER> and <IUSR> with full accees on the folder c:\Windows\Temp. But, not success.

    An operation attempted against a FAST endpoint exprienced an exception. This operation may be retried. Error details: Microsoft.Exchange.Search.Fast.PerformingFastOperationException: An Exception was received during a FAST operation. ---> System.ServiceModel.FaultException: error CS1548: Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' -- 'Error signing assembly -- Unknown error (8013141c)'

    Tuesday, November 12, 2019 10:31 PM
  • AHHH finaly.

    The CU3 make anything stranger...

    I started doing some digging that pointed me at C:\Users\All Users\Microsoft\Crypto\RSA.

    I renamed this folder...

    AND run this:

    Invoke-Command -ComputerName SRVEX001 {Stop-Service MSExchangeFastSearch; Stop-Service HostControllerService}

    AND this

    Invoke-Command -ComputerName SRVEX001 {Start-Service MSExchangeFastSearch; Start-Service HostControllerService}

    Now the search service is ok.

    Very stranger the CU3 to do this...

    Tuesday, November 12, 2019 10:49 PM
  • Hi EvandroMalmsteen,

    It's great that you have solved the search issue, and thanks for your sharing. You can mark your reply as answer, his may help more people with similar problems. Here is a brief summary about this issue.

    Issue Symptom

    ====================================

    The Exchange Server 2019 CU3 is with problem on the Search Index. The users can't searching in OWA or Outlook app. The error is "Time out for test thread".

    The logs in event viewer is:

    An operation attempted against a FAST endpoint exprienced an exception. This operation may be retried. Error details: Microsoft.Exchange.Search.Fast.PerformingFastOperationException: An Exception was received during a FAST operation. ---> System.ServiceModel.FaultException: error CS1548: Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' -- 'Error signing assembly -- Unknown error (8013141c)'

    Possible Cause

    ====================================

    From the error information "Cryptographic failure while signing assembly 'c:\Windows\Temp\xhlog4pq.g3p\Microsoft.Exchange.BigFunnelFlow.28.dll' ", this issue may have relation with cryptographic key.

    Solution Summary

    ====================================

    Rename the folder MachineKeys under C:\Users\All Users\Microsoft\Crypto\RSA. Then restart Microsoft Exchange Search and Microsoft Exchange Search Host Controller services.

    Invoke-Command -ComputerName <name> {Stop-Service MSExchangeFastSearch; Stop-Service HostControllerService} Invoke-Command -ComputerName <name> {Start-Service MSExchangeFastSearch; Start-Service HostControllerService}

    Regards,

    Lydia Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Wednesday, November 13, 2019 6:42 AM
    Moderator