locked
Unable to manager Failover Cluster Manager in SQL 2008 R2 Windows Cluster RRS feed

  • Question

  • Hi,

    We have an issue with a SQL 2008 R2 windows cluster (Windows 2008 R2 x64) where over a period of time, we lose connectivity to the Failover Cluster manager tool. the only noticable error found in the logs was the following:

    The following errors occur in system logs: (event id 1053)

    The processing of Group Policy failed. Windows could not resolve the user name. This could be caused by one of more of the following:

    a) Name Resolution failure on the current domain controller.

    b) Active Directory Replication Latency (an account created on another domain controller has not replicated to the current domain controller).

    I have noted the following errors in the event logs:

    Failover Cluster Manager could not contact node 'NODE NAME'.

    System.ApplicationException: Failed to open the event session. ---> System.ComponentModel.Win32Exception: The RPC server is unavailable
       --- End of inner exception stack trace ---

    Server stack trace: 
       at MS.Internal.ServerClusters.EventLogSession.OpenSession(String server, Int32 timeout)
       at MS.Internal.ServerClusters.EventLogSession..ctor(String serverName)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.<>c__DisplayClass3.<QueryWorker>b__0()
       at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
       at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

    Exception rethrown at [0]: 
       at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
       at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.AsyncCallDelegate`1.EndInvoke(IAsyncResult result)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.ExecuteAsyncCall[T](AsyncCallDelegate`1 asyncCall)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.QueryWorker(Object a)

    We have tackled Errornous DNS entries to be removed, NIC binding orders modified, Netbackup client updated to 7.1.0.4, nic drivers updated, firmware updated but no success. Can anyone advise how root cause could be established in this scenario?

    Many thanks.

    deemy.

    Failover Cluster Manager could not contact node 'Cluster node name'.

    System.ApplicationException: Failed to open the event session. ---> System.ComponentModel.Win32Exception: The RPC server is unavailable
       --- End of inner exception stack trace ---

    Server stack trace: 
       at MS.Internal.ServerClusters.EventLogSession.OpenSession(String server, Int32 timeout)
       at MS.Internal.ServerClusters.EventLogSession..ctor(String serverName)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.<>c__DisplayClass3.<QueryWorker>b__0()
       at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
       at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

    Exception rethrown at [0]: 
       at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
       at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.AsyncCallDelegate`1.EndInvoke(IAsyncResult result)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.ExecuteAsyncCall[T](AsyncCallDelegate`1 asyncCall)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.QueryWorker(Object a)

    Failover Cluster Manager could not contact node 'Cluster node name'.

    System.ApplicationException: Failed to open the event session. ---> System.ComponentModel.Win32Exception: The RPC server is unavailable
       --- End of inner exception stack trace ---

    Server stack trace: 
       at MS.Internal.ServerClusters.EventLogSession.OpenSession(String server, Int32 timeout)
       at MS.Internal.ServerClusters.EventLogSession..ctor(String serverName)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.<>c__DisplayClass3.<QueryWorker>b__0()
       at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
       at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

    Exception rethrown at [0]: 
       at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
       at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.AsyncCallDelegate`1.EndInvoke(IAsyncResult result)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.ExecuteAsyncCall[T](AsyncCallDelegate`1 asyncCall)
       at MS.Internal.ServerClusters.Management.EventLogQuerySet.QueryWorker(Object a)

    Thursday, May 9, 2013 4:43 PM

Answers

  • First thing is to make sure all the DNS and AD entries for the cluster have not been deleted. 

    Geoff N. Hiten Principal Consultant Microsoft SQL Server MVP

    • Proposed as answer by Fanny Liu Tuesday, May 14, 2013 10:23 AM
    • Marked as answer by Fanny Liu Wednesday, May 15, 2013 10:33 AM
    Thursday, May 9, 2013 4:51 PM
  • Hello,

    The following blog is about troubleshoot cluster node connection issue, please refer to:

    http://blogs.msdn.com/b/clustering/archive/2010/11/23/10095621.aspx

    Regards,
    Fanny Liu


    Fanny Liu
    TechNet Community Support

    • Proposed as answer by Fanny Liu Tuesday, May 14, 2013 10:23 AM
    • Marked as answer by Fanny Liu Wednesday, May 15, 2013 10:33 AM
    Friday, May 10, 2013 5:57 AM

All replies

  • First thing is to make sure all the DNS and AD entries for the cluster have not been deleted. 

    Geoff N. Hiten Principal Consultant Microsoft SQL Server MVP

    • Proposed as answer by Fanny Liu Tuesday, May 14, 2013 10:23 AM
    • Marked as answer by Fanny Liu Wednesday, May 15, 2013 10:33 AM
    Thursday, May 9, 2013 4:51 PM
  • Hello,

    The following blog is about troubleshoot cluster node connection issue, please refer to:

    http://blogs.msdn.com/b/clustering/archive/2010/11/23/10095621.aspx

    Regards,
    Fanny Liu


    Fanny Liu
    TechNet Community Support

    • Proposed as answer by Fanny Liu Tuesday, May 14, 2013 10:23 AM
    • Marked as answer by Fanny Liu Wednesday, May 15, 2013 10:33 AM
    Friday, May 10, 2013 5:57 AM