locked
SCOM 2012 UR3 - Console error - stack overflow RRS feed

  • Question

  • It seems that ever since I've applied UR3, I've been receiving errors when using the Operations Console to author Distributed Applications and Groups.  This is rather annoying, what can be done?

    When creating/modifying a group:

    Date: 10/23/2012 10:11:49 AM

    Application: Operations Manager

    Application Version: 7.0.8560.0

    Severity: Error

    Message:

    Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:

       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()

       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:

       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)

       --- End of inner exception stack trace ---

       at Microsoft.EnterpriseManagement.Common.Internal.ExceptionHandlers.HandleChannelExceptions(Exception ex)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.HandleChannelExceptions(String methodName, Exception ex)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)

       at Microsoft.EnterpriseManagement.Mom.Internal.UI.Console.ConsoleJobExceptionHandler.ExecuteJob(IComponent component, EventHandler`1 job, Object sender, ConsoleJobEventArgs args)

    System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:

       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()

       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:

       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)

    When creating or editing a Distributed Application:

    Date: 10/20/2012 7:03:27 AM

    Application: Operations Manager

    Application Version: 7.0.8560.0

    Severity: Warning

    Message:

    Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:

       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()

       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:

       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)

       --- End of inner exception stack trace ---

       at Microsoft.EnterpriseManagement.Common.Internal.ExceptionHandlers.HandleChannelExceptions(Exception ex)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.HandleChannelExceptions(String methodName, Exception ex)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)

       at Microsoft.EnterpriseManagement.Mom.Internal.UI.Console.ConsoleJobExceptionHandler.ExecuteJob(IComponent component, EventHandler`1 job, Object sender, ConsoleJobEventArgs args)

    System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:

       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()

       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:

       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)

       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)


    B. Wright

    Tuesday, October 23, 2012 2:22 PM

Answers

  • I found this to be a bug in one of my custom management packs.  Slowly peeling back the onion by deleting groups, DAs, overrides, etc and removing dependencies, reimporting I was able to clear up the error.  It looks like something inadvertently was placed into the 'Default management pack' - not nice things... live and learn.

    B. Wright

    • Marked as answer by B. Wright Friday, December 21, 2012 9:22 AM
    Friday, December 21, 2012 9:21 AM

All replies

  • Hi,

    Try with the /clearcache parameter - "C:\Program Files\System Center 2012\Operations Manager\Console\Microsoft.EnterpriseManagement.Monitoring.Console.exe" /clearcache

    Or check here:

    Error: ‘The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection.’

    http://thoughtsonopsmgr.blogspot.dk/2009/11/error-client-has-been-disconnected-from.html

    Attempts to opening the System Center Operations Manager 2007 console fails with "The client has been disconnected from the server" exception

    http://support.microsoft.com/kb/2262476

    SCOM2012: Error when creating a new dashboard

    http://www.systemcentercentral.com/Forums/tabid/60/indexId/93579/Default.aspx?tag=Forums+Operations_Manager#vindex95507


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    Tuesday, October 23, 2012 2:56 PM
  • /clearcache - nothing.  I've referenced the "thoughtsonopsmgr" blog, and tried all the troubleshooting there, and no dice.

    One comment has promise though, but I don't know how to find what MP may have the issue and/or needs to be reimported?

    Lynne "Tiger" Taggart  said...

    I ran into the same issue a while ago and although I ran through these steps, it turned out that I was in fault. I removed some Management packs, and it turned out that one or more was part of the default management pack. Once I re-imported the default MP, my error was gone

    The other link refers to a hotfix for 2007.  I'm using SCOM 2012 UR3 on 2008 R2 SP1.

    Is there anyway that I can fix the MP corruption that I may have?  I've never put any of my custom objects into the "default management pack", but I have several 'custom' ones that I backup religously (backupmps.ps1), and have imported many from MS's catalog.


    B. Wright

    Tuesday, October 23, 2012 3:24 PM
  • Hi,

    Any errors in the event viewer? Is it possible to restart the server(s) ?


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    Tuesday, October 23, 2012 3:42 PM
  • Tuesday, October 23, 2012 4:05 PM
  • That's correct, I have no time issues.  the client, RMS, other mgt servers, and DCs are all in sync to the second.  Nothing shows up on the Application or Operations Manager event logs, and I've bounced the servers several times. My management group shows completely healthy, and everything seems to work OK, except upon saving the new or edited group/distributed application.  I get this error no matter where I run the console from.

    B. Wright

    Tuesday, October 23, 2012 4:12 PM
  • OK, then check if UR3 is applied correctly http://blogs.technet.com/b/stefan_stranger/archive/2012/10/22/opsmgr-2012-update-rollup-3-ships-and-my-experience-installing-it.aspx
    • Proposed as answer by Nicholas Li Monday, October 29, 2012 8:59 AM
    • Unproposed as answer by B. Wright Tuesday, October 30, 2012 2:21 PM
    Tuesday, October 23, 2012 4:28 PM
  • Well, I've done the above, and every checkpoint now shows UR3 is installed in full, and correctly yet it did not solve my issue.  I still get the below error every time I try to save a new, or edit an existing Distributed Application:

    Two Dialog

    "The client has been disconnected from the server.  Please call ManagementGroup.Reconnect () to reestablish the connection"

    "Failed to populate membership and relationship instances"

    Date: 10/30/2012 10:18:37 AM
    Application: Operations Manager
    Application Version: 7.0.8560.0
    Severity: Warning
    Message:

    Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:
       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)
       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)
       --- End of inner exception stack trace ---
       at Microsoft.EnterpriseManagement.Common.Internal.ExceptionHandlers.HandleChannelExceptions(Exception ex)
       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.HandleChannelExceptions(String methodName, Exception ex)
       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)
       at Microsoft.EnterpriseManagement.Mom.Internal.UI.Console.ConsoleJobExceptionHandler.ExecuteJob(IComponent component, EventHandler`1 job, Object sender, ConsoleJobEventArgs args)
    System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

    Server stack trace:
       at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.DispatchUnknownMessage(Message message)
       at Microsoft.EnterpriseManagement.Common.Internal.EntityObjectsServiceProxy.PerformGroupMembershipDiscoverySnapshot(String managementPackXml)


    B. Wright

    Tuesday, October 30, 2012 2:21 PM
  • Sill, in all my research no one has a fix.  Any other ideas?  Time sync is good, UR3 installed on all management servers.  Opening every console w/ clearcache, yet everytime I edit a group or dist. app I get this error.

    B. Wright

    Saturday, December 8, 2012 8:46 AM
  • Hi

    I don't think there is an easy answer - you've covered the basics and more. I assume that this happens from every SCOM console?

    I suspect it is time to either:

    1) Open a PSS ticket

    2) Set up a lab environment - if you have Veeam for backup then you could use their Virtual Sandbox technology - then take a copy of production and start removing management packs one at a time. And see if there is an issue with one that is causing this. 

    Cheers

    Graham 


    Regards Graham New System Center 2012 Blog! - http://www.systemcentersolutions.co.uk
    View OpsMgr tips and tricks at http://systemcentersolutions.wordpress.com/

    Saturday, December 8, 2012 9:02 AM
  • I found this to be a bug in one of my custom management packs.  Slowly peeling back the onion by deleting groups, DAs, overrides, etc and removing dependencies, reimporting I was able to clear up the error.  It looks like something inadvertently was placed into the 'Default management pack' - not nice things... live and learn.

    B. Wright

    • Marked as answer by B. Wright Friday, December 21, 2012 9:22 AM
    Friday, December 21, 2012 9:21 AM
  • Hello, 

    I had the same issue while pushing agent on one of our servers. Upon troubleshooting I found out that, the agent on that server was installed manually and was not approved from pending management.

    Upon un-installing the agent from that server, I was successfully able to push it from the console, without facing a problem.

    Hope this helps someone.

    Regards,


    Abdul Karim. (http://sites.google.com/site/scomblogs Twitter:@Abdul_SCOM)

    Tuesday, May 7, 2013 9:22 AM