locked
Excel serveices app Pool crashing frequently RRS feed

  • Question

  • Hi,

    I am facing a peculiar issue in our sharePoint 2013 environment wherein  Excel services app pool is constantly crashing and thee errors are shown in the EventLog

    Entry in the Event Log

    Application: w3wp.exe
    Framework Version: v4.0.30319
    Description: The application requested process termination through System.Environment.FailFast(string message).
    Message:
    Stack:
       at System.Environment.FailFast(System.String)
       at Microsoft.Office.Excel.Server.HelperMethods.HandleShutdown(System.Exception, Boolean)
       at Microsoft.Office.Server.Diagnostics.FirstChanceHandler.ExceptionFilter(Boolean, TryBlock, FilterBlock, CatchBlock, FinallyBlock)
       at Microsoft.Office.Web.Common.ExceptionReporting.ReportOnExceptionTag(UInt32, Category, System.String, Boolean, Microsoft.Office.Web.Common.TryBlock, Microsoft.Office.Web.Common.CatchBlock, Microsoft.Office.Web.Common.FinallyBlock)
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServerApp..ctor()
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServerApp..cctor()
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.SafeSetContext()
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.BeginProcessOperation(Microsoft.Office.Excel.Server.CalculationServer.CommandParameter, Microsoft.Office.Excel.Server.WebMethodBehaviorAttribute, Microsoft.Office.Excel.Server.WebMethodType, System.AsyncCallback, System.Object, Microsoft.Office.Excel.Server.CalculationServer.Operations.UserOperation)
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.BeginProcessOperation(Microsoft.Office.Excel.Server.CalculationServer.CommandParameter, Microsoft.Office.Excel.Server.WebMethodType, System.AsyncCallback, System.Object, Microsoft.Office.Excel.Server.CalculationServer.Operations.UserOperation)
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceSoap.BeginGetHealthScore(System.AsyncCallback, System.Object)
       at DynamicClass.AsyncInvokeBeginBeginGetHealthScore(System.Object, System.Object[], System.AsyncCallback, System.Object)
       at System.ServiceModel.Dispatcher.AsyncMethodInvoker.InvokeBegin(System.Object, System.Object[], System.AsyncCallback, System.Object)
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper+<>c__DisplayClass7.<InvokeBegin>b__6()
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper.InvokeHelper(InvokeMethodDelegate, System.Reflection.MethodInfo)
       at Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper.InvokeBegin(System.Object, System.Object[], System.AsyncCallback, System.Object)
       at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(System.ServiceModel.Dispatcher.MessageRpc ByRef)
       at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef)
       at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(System.ServiceModel.Dispatcher.MessageRpc ByRef)
       at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)
       at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)
       at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)
       at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)
       at System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
       at System.Runtime.AsyncResult.Complete(Boolean)
       at System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].InnerTryReceiveCompletedCallback(System.IAsyncResult)
       at System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
       at System.Runtime.AsyncResult.Complete(Boolean)
       at System.Runtime.InputQueue`1+AsyncQueueReader[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Set(Item<System.__Canon>)
       at System.Runtime.InputQueue`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].EnqueueAndDispatch(Item<System.__Canon>, Boolean)
       at System.Runtime.InputQueue`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].EnqueueAndDispatch(System.__Canon, System.Action, Boolean)
       at System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Enqueue(System.__Canon, System.Action, Boolean)
       at System.ServiceModel.Channels.HttpPipeline+EnqueueMessageAsyncResult.CompleteParseAndEnqueue(System.IAsyncResult)
       at System.ServiceModel.Channels.HttpPipeline+EnqueueMessageAsyncResult.HandleParseIncomingMessage(System.IAsyncResult)
       at System.Runtime.AsyncResult.SyncContinue(System.IAsyncResult)
       at System.ServiceModel.Channels.HttpPipeline+EmptyHttpPipeline.BeginProcessInboundRequest(System.ServiceModel.Channels.ReplyChannelAcceptor, System.Action, System.AsyncCallback, System.Object)
       at System.ServiceModel.Channels.HttpChannelListener`1+HttpContextReceivedAsyncResult`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].ProcessHttpContextAsync()
       at System.ServiceModel.Channels.HttpChannelListener`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].BeginHttpContextReceived(System.ServiceModel.Channels.HttpRequestContext, System.Action, System.AsyncCallback, System.Object)
       at System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult)
       at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
       at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
       at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(System.Object)
       at System.ServiceModel.AspNetPartialTrustHelpers.PartialTrustInvoke(System.Threading.ContextCallback, System.Object)
       at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequestWithFlow(System.Object)
       at System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
       at System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
       at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)

    Another entry in the event Log
    -----------------------------------------------------------------------------
    Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp: 0x5215df96
    Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
    Exception code: 0x80131623
    Fault offset: 0x00007ffc8d12a323
    Faulting process id: 0x1428
    Faulting application start time: 0x01d28094a106f92d
    Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
    Faulting module path: unknown
    Report Id: 8e2fdda9-ecab-11e6-80ed-aa000019ac15
    Faulting package full name:
    Faulting package-relative application ID:

    I have followed the steps given @ this link https://blogs.msdn.microsoft.com/spike/2015/02/12/sharepoint-2013-excel-services-w3wp-exe-fails-with-exception-code-0x80131623/  but it did not helped me.

    Regards

    Rajaniesh

    Monday, February 6, 2017 11:28 PM

All replies

  • Hi,

    I used windbg to catch the real cause and here is my finding

    0:000> !pe
    Exception object: 000000a27e373888
    Exception type:   System.TypeInitializationException
    Message:          The type initializer for 'Microsoft.Office.Excel.Server.CalculationServer.ExcelServerApp' threw an exception.
    InnerException:   System.AccessViolationException, Use !PrintException 000000a27e368e18 to see more.
    StackTrace (generated):
        SP               IP               Function
        0000009FFDB7D7B0 00007FFE71977E58 Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.SafeSetContext()+0x18
        0000009FFDB7D7E0 00007FFE71976CD2 Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.BeginProcessOperation(Microsoft.Office.Excel.Server.CalculationServer.CommandParameter, Microsoft.Office.Excel.Server.WebMethodBehaviorAttribute, Microsoft.Office.Excel.Server.WebMethodType, System.AsyncCallback, System.Object, Microsoft.Office.Excel.Server.CalculationServer.Operations.UserOperation)+0x3a2
        0000009FFDB7D950 00007FFE719768FE Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceBase.BeginProcessOperation(Microsoft.Office.Excel.Server.CalculationServer.CommandParameter, Microsoft.Office.Excel.Server.WebMethodType, System.AsyncCallback, System.Object, Microsoft.Office.Excel.Server.CalculationServer.Operations.UserOperation)+0x6e
        0000009FFDB7D9C0 00007FFE719767AF Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceSoap.BeginGetHealthScore(System.AsyncCallback, System.Object)+0x6f
        0000009FFDB7DA20 00007FFE71C020FE System_ServiceModel_ni!DynamicClass.AsyncInvokeBeginBeginGetHealthScore(System.Object, System.Object[], System.AsyncCallback, System.Object)+0x5e
        0000009FFDB7DA60 00007FFEC45661A9 System_ServiceModel_ni!System.ServiceModel.Dispatcher.AsyncMethodInvoker.InvokeBegin(System.Object, System.Object[], System.AsyncCallback, System.Object)+0x389
        0000009FFDB7DB90 00007FFE71976724 Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper+<>c__DisplayClass7.<InvokeBegin>b__6()+0x34
        0000009FFDB7DBD0 00007FFE719765CA Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper.InvokeHelper(InvokeMethodDelegate, System.Reflection.MethodInfo)+0x3a
        0000009FFDB7DC30 00007FFE71976557 Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServiceHostFactory+OperationInvokerExceptionWrapper.InvokeBegin(System.Object, System.Object[], System.AsyncCallback, System.Object)+0x97
        0000009FFDB7DC80 00007FFEC45C2A0C System_ServiceModel_ni!System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x66c
        0000009FFDB7E030 00007FFEC45C20A0 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x110
        0000009FFDB7E0E0 00007FFEC45C178B System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x17b
        0000009FFDB7E2C0 00007FFEC45C01CE System_ServiceModel_ni!System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)+0x17e
        0000009FFDB7E380 00007FFEC45BF71E System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)+0x7ee
        0000009FFDB7E760 00007FFEC45BDB42 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)+0x1e2
        0000009FFDB7E7E0 00007FFEC45BD68B System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)+0xdb
        0000009FFDB7E840 00007FFEC0C2ACC2 System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+0x32
        0000009FFDB7E890 00007FFEC0C2580A System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+0x17a
        0000009FFDB7E940 00007FFEC50F18BC System_ServiceModel_ni!System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].InnerTryReceiveCompletedCallback(System.IAsyncResult)+0xcc
        0000009FFDB7E9B0 00007FFEC0C2ACC2 System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+0x32
        0000009FFDB7EA00 00007FFEC0C2580A System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+0x17a
        0000009FFDB7EAB0 00007FFEC0C2AEE1 System_ServiceModel_Internals_ni!System.Runtime.InputQueue`1+AsyncQueueReader[[System.__Canon, mscorlib]].Set(Item<System.__Canon>)+0x41
        0000009FFDB7EB00 00007FFEC0C2A80C System_ServiceModel_Internals_ni!System.Runtime.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(Item<System.__Canon>, Boolean)+0x3dc
        0000009FFDB7EC70 00007FFEC0C2A3DE System_ServiceModel_Internals_ni!System.Runtime.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(System.__Canon, System.Action, Boolean)+0x7e
        0000009FFDB7ECE0 00007FFEC45E9072 System_ServiceModel_ni!System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Enqueue(System.__Canon, System.Action, Boolean)+0x62
        0000009FFDB7ED40 00007FFEC45E8A0E System_ServiceModel_ni!System.ServiceModel.Channels.HttpPipeline+EnqueueMessageAsyncResult.CompleteParseAndEnqueue(System.IAsyncResult)+0x17e
        0000009FFDB7EDC0 00007FFEC45E887A System_ServiceModel_ni!System.ServiceModel.Channels.HttpPipeline+EnqueueMessageAsyncResult.HandleParseIncomingMessage(System.IAsyncResult)+0x3a
        0000009FFDB7EDF0 00007FFEC0CBFDA4 System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.SyncContinue(System.IAsyncResult)+0x97254
        0000009FFDB7EE40 00007FFEC45E7820 System_ServiceModel_ni!System.ServiceModel.Channels.HttpPipeline+EmptyHttpPipeline.BeginProcessInboundRequest(System.ServiceModel.Channels.ReplyChannelAcceptor, System.Action, System.AsyncCallback, System.Object)+0xe0
        0000009FFDB7EE90 00007FFEC45E6F18 System_ServiceModel_ni!System.ServiceModel.Channels.HttpChannelListener`1+HttpContextReceivedAsyncResult`1[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].ProcessHttpContextAsync()+0x1e8
        0000009FFDB7EF20 00007FFEC45E6D0A System_ServiceModel_ni!System.ServiceModel.Channels.HttpChannelListener`1[[System.__Canon, mscorlib]].BeginHttpContextReceived(System.ServiceModel.Channels.HttpRequestContext, System.Action, System.AsyncCallback, System.Object)+0x8a
        0000009FFDB7EF80 00007FFEBA72E02C System_ServiceModel_Activation_ni!System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult)+0x15c
        0000009FFDB7F020 00007FFEBA7282B9 System_ServiceModel_Activation_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()+0x249
        0000009FFDB7F0B0 00007FFEBA727F95 System_ServiceModel_Activation_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()+0x55
        0000009FFDB7F160 00007FFEBA727F23 System_ServiceModel_Activation_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(System.Object)+0x53
        0000009FFDB7F1B0 00007FFEBA6E8952 System_ServiceModel_Activation_ni!System.ServiceModel.AspNetPartialTrustHelpers.PartialTrustInvoke(System.Threading.ContextCallback, System.Object)+0x72
        0000009FFDB7F200 00007FFEBA727EA3 System_ServiceModel_Activation_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequestWithFlow(System.Object)+0xf3
        0000009FFDB7F260 00007FFEC0C2BF11 System_ServiceModel_Internals_ni!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x71
        0000009FFDB7F2C0 00007FFEC0C2BE90 System_ServiceModel_Internals_ni!System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x40
        0000009FFDB7F320 00007FFECFD48DE6 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x96

    StackTraceString: <none>
    HResult: 80131534

    0:000> !PrintException 000000a27e368e18
    Exception object: 000000a27e368e18
    Exception type:   System.AccessViolationException
    Message:          Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
    InnerException:   <none>
    StackTrace (generated):
        SP               IP               Function
        0000009FFDB78190 0000000000000001 mscorlib_ni!System.Environment.FailFast(System.String)+0x2
        0000009FFDB78190 00007FFE72BC7B34 Microsoft_Office_Excel_Server!Microsoft.Office.Excel.Server.HelperMethods.HandleShutdown(System.Exception, Boolean)+0x284
        0000009FFDB78220 00007FFE71BB03A6 Microsoft_Office_Server_Diagnostics!Microsoft.Office.Server.Diagnostics.FirstChanceHandler.ExceptionFilter(Boolean, TryBlock, FilterBlock, CatchBlock, FinallyBlock)+0xe6
        0000009FFDB7C7C0 00007FFE71BAFE88 Microsoft_Office_Web_Common!Microsoft.Office.Web.Common.ExceptionReporting.ReportOnExceptionTag(UInt32, Category, System.String, Boolean, Microsoft.Office.Web.Common.TryBlock, Microsoft.Office.Web.Common.CatchBlock, Microsoft.Office.Web.Common.FinallyBlock)+0x88
        0000009FFDB7C830 00007FFE719780E1 Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServerApp..ctor()+0x1d1
        0000009FFDB7C8A0 00007FFE71977EDB Microsoft_Office_Excel_Server_CalculationServer!Microsoft.Office.Excel.Server.CalculationServer.ExcelServerApp..cctor()+0x4b

    StackTraceString: <none>
    HResult: 80004003

    Microsoft team, Any clue why ExcelServerApp constructor is raising this exception.

    Regards
    Rajaniesh

    Tuesday, February 7, 2017 2:29 PM
  • Can somebody reply ?
    Wednesday, February 8, 2017 3:38 AM