none
File share witness issue with the DAG RRS feed

  • Question

  • I have a 2-member DAG and 2-member CA/HT CAS array Exchange 2010 system. CA1 server is designated as the primary DAG witness, CA2 is the alternate witness.

    At one time I had used the third CA server CA3 as one of the file share witnesses. This server, CA3, has been powered off for a few months, but not yet removed from the Exchange system.

    Even though CA3 is no longer a designated witness, (and it’s not even powered on), I still keep getting the following error with Event ID 1564:

    File share witness resource 'File Share Witness (\\ca3.MyOrg.org\DAG.MyOrg.org)' failed to arbitrate for the file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster.

    Occasionally my DAG is down for some unknown reason, causing mailboxes inaccessible, and I am wondering if the above event could be the cause. How can I fix this?

    Thanks and regards.

    Wednesday, December 5, 2012 3:52 PM

Answers

  • If you type this, what does it return ?

    cluster res "File Share Witness" /priv


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Looks like we need some modification in this command.

    Cluster res "File Share Witness (\\Server name\fsw_folder name)" /priv /delete

    In your case the command will be like this.

    Cluster res "File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org)" /delete


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**


    • Edited by Om Prakash Nath Saturday, December 8, 2012 8:50 AM modify
    • Marked as answer by Zi Feng Monday, December 17, 2012 1:58 AM
    Saturday, December 8, 2012 8:31 AM
  • This really started as troubleshooting a DAG issue, which sometimes brings my Exchange down. If Windows or Exchange cannot "tell it like it is" for the FSW, how can I rule it out as a potential root cause? Again, this is the event:

    File share witness resource 'File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org.' failed a periodic health check on file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster. (Note, caht4 is powered off).

    There could be a problem with the cluster. But it may not necessarily because the FSW is on the wrong server.

    You mentioned caht4 is down but the FSW resource is online. This is good enough to conclude it's not using caht4. A name is just a name. E.g. one can name a mailbox server as CAS01. It doesn't mean server CAS01 must be holding CAS role.

    You can further confirm it by shutting down caht1 and you can see the FSW is offline.

    • Marked as answer by Zi Feng Monday, December 17, 2012 1:58 AM
    Saturday, December 8, 2012 10:30 AM
  • This really started as troubleshooting a DAG issue, which sometimes brings my Exchange down. If Windows or Exchange cannot "tell it like it is" for the FSW, how can I rule it out as a potential root cause? Again, this is the event:

    File share witness resource 'File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org.' failed a periodic health check on file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster. (Note, caht4 is powered off).

    The Display Name issue is known one if the FSW was changed at some point. You can just go in the FSW value in the Failover Cluster Manager and edit the name if you want. (thats what I always do!)

    http://blogs.technet.com/b/timmcmic/archive/2010/07/13/exchange-2010-file-share-witness-oddities.aspx

    Now if you are getting failure warnings for the FSW, thats a different issue. I would consider simply moving the cluster to see if that solves it:

    cluster group "Cluster Group" /move

    or reboot the FSW.

    If you have done these steps already I apologize, I didnt read back through each post in the thread  :)




    Saturday, December 8, 2012 12:46 PM
    Moderator

All replies

  • Run this command without any switch, it will set your FSW to CA1 fixing any issue with configuration.

    Set-DatabaseAvailabilityGroup -Identity DAG1


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Wednesday, December 5, 2012 6:45 PM
  • http://technet.microsoft.com/en-us/library/cc756221(v=ws.10).aspx

    I assume you have checked the above?

    Check again to  see the quorum type


    Sukh

    Wednesday, December 5, 2012 6:56 PM
  • Hi Fat Frog

    Any update, did you try from above?

    Cheers


    Zi Feng
    TechNet Community Support

    Thursday, December 6, 2012 7:46 AM
  • I thought DAG should only be managed via the Exchange Management Console, not the Windows Failover Cluster Manager.

    For my DAG under the Windows FCM, I can only see info under "Node" and "Network"; whereas I see nothing under "applications and services" and "storage". (This is different from a SQL cluster under FCM).

    Is this the way it's supposed to be?

    Thanks and regards.

    Friday, December 7, 2012 6:08 AM
  • I did what you suggested in this format, "Set-DatabaseAvailabilityGroup -Identity DAG", as my DAG's dns name is just DAG. I cannot confirm the effect now, because event 1564 doesn't happen daily.

    However I am still seeing event 4002 that still mentions my offline CA servers.

    Process 4732: ProxyWebRequest CrossSite from S-1-1-0 to https://caht3.MyOrg.org/EWS/Exchange.asmx failed. Caller SIDs: NetworkCredentials. The exception returned is Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: System.Net.WebException: The remote name could not be resolved: 'caht3.myOrg.org' at System.Web.Services.Protocols.WebClientAsyncResult.WaitForResponse()

    ..... (trauncated the rest of the event details for this posting).

    Thanks and regards.

    Friday, December 7, 2012 6:19 AM
  • http://blogs.technet.com/b/timmcmic/archive/2012/03/12/verifying-the-file-share-witness-server-directory-in-use-for-exchange-2010.aspx

    Review this KB and check if are are mapped to correct FSW.

    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Friday, December 7, 2012 7:08 AM
  • What do you see, post a screenshot, it should show you the quorum type?


    Sukh

    Friday, December 7, 2012 9:44 AM
  • Following the link provided by Om Prakash Nath, I ran this, and the correct witness server names are returned.

    This shows the correct witness server names

    I then ran this, and the WRONG witness server is returned, because caht4 is no longer designated as a witness server, and it's not even powered on.

     

    I don't know from where it gets the wrong server, because when I ran the quorum wizard and spedify caht5, it says caht5 has already been set as the witness, therefore no change will be made.

    Thanks and regards. 


    • Edited by Fat Frog Friday, December 7, 2012 10:59 PM clarification
    Friday, December 7, 2012 10:57 PM
  • Delete this failed FSW from Cluster configuration.

    cluster res "File Share Witness" /delete

    Verify by running the previous cluster command that it's been removed.


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Saturday, December 8, 2012 5:11 AM
  • I thought DAG should only be managed via the Exchange Management Console, not the Windows Failover Cluster Manager.

    For my DAG under the Windows FCM, I can only see info under "Node" and "Network"; whereas I see nothing under "applications and services" and "storage". (This is different from a SQL cluster under FCM).

    Is this the way it's supposed to be?

    Thanks and regards.

    Yes, you're right. DAG cluster should be managed via EMC or EMS, not cluster console. But...

    You CAN review the result via the cluster console. In this case, you can expand cluster core resource from cluster console. If you see file share witness pointed to CAS1, it is configured properly.

    Saturday, December 8, 2012 5:24 AM
  • Following the link provided by Om Prakash Nath, I ran this, and the correct witness server names are returned.

    This shows the correct witness server names

    I then ran this, and the WRONG witness server is returned, because caht4 is no longer designated as a witness server, and it's not even powered on.

     

    I don't know from where it gets the wrong server, because when I ran the quorum wizard and spedify caht5, it says caht5 has already been set as the witness, therefore no change will be made.

    Thanks and regards. 


    The result indicates that FSW is NOT configured properly.

    This cmdlet will correct it.

    Set-DatabaseAvailbbilityGroup -Identity DAG

    Saturday, December 8, 2012 5:29 AM
  • This has already been run, as suggesting in the beginning, but it didn't fix the problem. And I ran it again, (the screen did flash the correct value, but running "cluster res", it is still showing the wrong value. (Frustrating).

    Saturday, December 8, 2012 6:10 AM
  • Delete this failed FSW from Cluster configuration.

    cluster res "File Share Witness" /delete

    Verify by running the previous cluster command that it's been removed.


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    For some reason, it errored out like this:

    Saturday, December 8, 2012 6:11 AM
  • If you type this, what does it return ?

    cluster res "File Share Witness" /priv


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Saturday, December 8, 2012 6:18 AM
  • Some times the FSW has been changed to point to the correct server but the name of the resource is not changed.

    Go to FCM, in the summary (top of the right panel), do you see the correct FSW server?

    Saturday, December 8, 2012 6:22 AM
  • It is throwing this error:

    [PS] C:\>cluster res "File Share Witness" /priv
    System error 5007 has occurred (0x0000138f).
    The cluster resource could not be found.

    • Proposed as answer by RUTKOVSKI Wednesday, August 28, 2013 7:00 PM
    Saturday, December 8, 2012 6:50 AM
  • The FCM does display the correct FSW server, however I get a lot of errors on my mailbox DAG servers that are still referencing the wrong server. 
    Saturday, December 8, 2012 6:52 AM
  • Go to FCM, expand the cluster core resource, do you see the FSW resource online or offline?
    Saturday, December 8, 2012 6:58 AM
  • Under the cluster core resource, it says it's online, but this extra server is actually being powered off.

    File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org)

    Saturday, December 8, 2012 7:11 AM
  • http://blogs.technet.com/b/timmcmic/archive/2010/07/13/exchange-2010-file-share-witness-oddities.aspx

    Follow this KB.

    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Saturday, December 8, 2012 7:19 AM
  • Under the cluster core resource, it says it's online, but this extra server is actually being powered off.

    File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org)

    Like what I said, it has been changed to point to the right server but the name was not changed.

    "File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org)"

    This is only the name of the resource. Not really a problem.

    • Edited by Li Zhen Saturday, December 8, 2012 7:24 AM
    Saturday, December 8, 2012 7:23 AM
  • This really started as troubleshooting a DAG issue, which sometimes brings my Exchange down. If Windows or Exchange cannot "tell it like it is" for the FSW, how can I rule it out as a potential root cause? Again, this is the event:

    File share witness resource 'File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org.' failed a periodic health check on file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster. (Note, caht4 is powered off).

    Saturday, December 8, 2012 7:44 AM
  • If you type this, what does it return ?

    cluster res "File Share Witness" /priv


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**

    Looks like we need some modification in this command.

    Cluster res "File Share Witness (\\Server name\fsw_folder name)" /priv /delete

    In your case the command will be like this.

    Cluster res "File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org)" /delete


    Om

    (MCITP,Enterprise Messaging Administrator)

    **My posts are provided “AS IS” without warranty of any kind**


    • Edited by Om Prakash Nath Saturday, December 8, 2012 8:50 AM modify
    • Marked as answer by Zi Feng Monday, December 17, 2012 1:58 AM
    Saturday, December 8, 2012 8:31 AM
  • This really started as troubleshooting a DAG issue, which sometimes brings my Exchange down. If Windows or Exchange cannot "tell it like it is" for the FSW, how can I rule it out as a potential root cause? Again, this is the event:

    File share witness resource 'File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org.' failed a periodic health check on file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster. (Note, caht4 is powered off).

    There could be a problem with the cluster. But it may not necessarily because the FSW is on the wrong server.

    You mentioned caht4 is down but the FSW resource is online. This is good enough to conclude it's not using caht4. A name is just a name. E.g. one can name a mailbox server as CAS01. It doesn't mean server CAS01 must be holding CAS role.

    You can further confirm it by shutting down caht1 and you can see the FSW is offline.

    • Marked as answer by Zi Feng Monday, December 17, 2012 1:58 AM
    Saturday, December 8, 2012 10:30 AM
  • This really started as troubleshooting a DAG issue, which sometimes brings my Exchange down. If Windows or Exchange cannot "tell it like it is" for the FSW, how can I rule it out as a potential root cause? Again, this is the event:

    File share witness resource 'File Share Witness (\\caht4.MyOrg.org\DAG.MyOrg.org.' failed a periodic health check on file share '\\caht1.MyOrg.org\DAG.MyOrg.org'. Please ensure that file share '\\caht1.MyOrg.org\DAG.MyOrg.org' exists and is accessible by the cluster. (Note, caht4 is powered off).

    The Display Name issue is known one if the FSW was changed at some point. You can just go in the FSW value in the Failover Cluster Manager and edit the name if you want. (thats what I always do!)

    http://blogs.technet.com/b/timmcmic/archive/2010/07/13/exchange-2010-file-share-witness-oddities.aspx

    Now if you are getting failure warnings for the FSW, thats a different issue. I would consider simply moving the cluster to see if that solves it:

    cluster group "Cluster Group" /move

    or reboot the FSW.

    If you have done these steps already I apologize, I didnt read back through each post in the thread  :)




    Saturday, December 8, 2012 12:46 PM
    Moderator