none
Server 2008 R2 SP1 cluster failover - Ownership of cluster disk unexpectedly lost

    Question

  • I am running SQL Server 2008 R2 on a Server 2008 R2 SP1 failover cluster. Cluster storage is provided by way of a Dell Equallogic PS 6000/6500 Array.

    Recently the cluster failed over without any apparent provocation. 


    Any help will be greatly appreciated, I will provide additional info as needed.




    The full cluster.log is located here:

    https://skydrive.live.com/redir?resid=A694FDEBF02727CD!126&authkey=!AEpuSOXKqYIphb0




    Here is an excerpt from the Windows System log and the cluster log. I believe this is where the trouble started:




    Log Name:      System
    Source:        Microsoft-Windows-FailoverClustering
    Date:          7/9/2012 8:17:10 AM
    Event ID:      1038
    Task Category: Physical Disk Resource
    Level:         Error
    Keywords:      
    User:          SYSTEM
    Computer:      SQLPRDSQLA.del.edu
    Description:
    Ownership of cluster disk 'Cluster Disk 1' has been unexpectedly lost by this node. Run the Validate a Configuration wizard to check your storage configuration.



    00001190.000020ac::2012/07/09-13:17:10.990 ERR   [RES] Physical Disk <Cluster Disk 1>: IsAlive sanity check failed!, pending IO completed with status 0.
    00001190.000020ac::2012/07/09-13:17:10.990 ERR   [RES] Physical Disk <Cluster Disk 1>: IsAlive sanity check failed!, pending IO completed with status 0.
    00001190.000020ac::2012/07/09-13:17:10.990 WARN  [RHS] Resource Cluster Disk 1 IsAlive has indicated failure.
    00001190.000020ac::2012/07/09-13:17:11.053 ERR   [RHS] Error 1726 calling RcmReceiveRhsReply().
    0000126c.00001270::2012/07/09-13:17:11.255 WARN  [RHS] Cluster service has terminated.
    0000126c.00001270::2012/07/09-13:17:11.255 INFO  [RHS] Terminating resource SQL Server Agent (state is Online).
    00001214.0000121c::2012/07/09-13:17:11.255 WARN  [RHS] Cluster service has terminated.
    00001044.00001048::2012/07/09-13:17:11.255 WARN  [RHS] Cluster service has terminated.
    00001190.00001194::2012/07/09-13:17:11.255 WARN  [RHS] Cluster service has terminated.
    00001214.0000121c::2012/07/09-13:17:11.255 INFO  [RHS] Terminating resource SQL Server (state is Online).
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource File Share Witness (state is Online).
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource SQL Network Name (MSSPRD) (state is Online).
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource Cluster Name (state is Online).
    00001190.000007b8::2012/07/09-13:17:11.271 INFO  [RES] Network Name <SQL Network Name (MSSPRD)>: Terminating resource...
    00001190.000007b8::2012/07/09-13:17:11.271 INFO  [RES] Network Name <SQL Network Name (MSSPRD)>: Offline of resource continuing...
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource Cluster Disk 2 (state is Online).
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource Cluster Disk 1 (state is Online).
    00001190.00001194::2012/07/09-13:17:11.271 INFO  [RHS] Terminating resource Cluster Disk 3 (state is Online).










    • Edited by gregs1329 Wednesday, July 11, 2012 6:12 PM
    Tuesday, July 10, 2012 8:16 PM

All replies

  • Hi,

    I have seen a similar issue before where it was indicative of an MPIO error.

    My advice would be to do the following.

    1. Un-install your MPIO (Multipath) software (Be sure to save settings especially in a cluster environment) and BACKUP License if applicable

    2. Reboot Host

    3. Remove Microsoft MPIO DSM (Guide: http://technet.microsoft.com/en-us/library/dd834839)

    4. Reboot Host

    5. Re-Install Microsoft MPIO DSM (Refer to above guide)

    6. Reboot Host

    7. Re-Install your MPIO (Multipath) software and associated licenses (If Any)

    8. Reboot Host

    9. Re-Scan for paths

    Hopefully this will resolve the issue.

    *Please note that the information above comes without guarantee or warranty. It is for informational purpose only.

    Kind Regards,

    Martin


    If you find my information useful, please rate it. :-)

    Wednesday, July 11, 2012 2:10 AM
  • Hi.

    Am I reading / understanding that right?

    You're using Cluster Shared Volumes (CSV) as disk resources for SQL-Server?

    Do you really mean Cluster Shared Volumes? Then this is only supported as Hyper-V Storage! http://technet.microsoft.com/en-us/library/dd630633(v=ws.10).aspx

    Or do you mean Volumes Shared by the Cluster? Then I didn't say anything. ;-)

    Regards

    Wednesday, July 11, 2012 3:21 PM
  • Oops... getting the terms mixed up I suppose. I have edited the original post accordingly.


    • Edited by gregs1329 Wednesday, July 11, 2012 4:00 PM
    Wednesday, July 11, 2012 4:00 PM
  • As I said: I didn't said anything ;-)
    Wednesday, July 11, 2012 5:43 PM