locked
win 2012 storage validation warning - "Validate CSV Settings" access denied RRS feed

  • Question

    • I'm trying to configure a lab environment to use clustering for hyper-v with 2 nodes. The validation test comes back with the following warnings during storage:

      Validating access using Server Message Block (SMB) protocol from node machine1.testlab.local to a share on node machine2.testlab.local.

      Failed to validate Server Message Block (SMB) share access through the IP address of the fault tolerant network driver for failover clustering (NetFT). The connection was attempted with the Cluster Shared Volumes test user account, from node machine1.testlab.local to the share on node machine2.testlab.local. Access is denied.

      Validating access using Server Message Block (SMB) protocol from node machine2.testlab.local to a share on node machine1.testlab.local.

      Failed to validate Server Message Block (SMB) share access through the IP address of the fault tolerant network driver for failover clustering (NetFT). The connection was attempted with the Cluster Shared Volumes test user account, from node machine2.testlab.local to the share on node machine1.testlab.local. Access is denied.

      I've tried looking online for ways around this but have come up empty. Any guidance? Thanks!


    • Edited by robdtec Wednesday, December 5, 2012 3:27 PM
    Wednesday, December 5, 2012 3:23 PM

Answers

  • Check following

    - File Server feature is installed on all cluster nodes.

    - SMB server is enabled on all cluster nodes, and in particular server service is running on all the nodes

    - Verify that your firewall is not blocking SMB outgoing or incomming traffic. Make sure that rules would not block it for any network/IP.

    - Check if from one machine you can access \\<machine name>\c$ on the other machine.


    Regards, Vladimir Petter, Microsoft Corporation

    • Marked as answer by Lawrence,Lu Friday, December 14, 2012 9:23 AM
    Wednesday, December 5, 2012 5:49 PM
  • I went line by line through all of our settings. It turns out someone higher in the AD forest was pushing down a registry change through group policy.

    They were disabling SMBv2 using

    Hive HKEY_LOCAL_MACHINE
    Key path SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
    Value name smb2
    Value type REG_DWORD
    Value data 0x0 (0)
    • Proposed as answer by uniken1 Tuesday, January 15, 2013 11:25 AM
    • Marked as answer by robdtec Thursday, January 31, 2013 1:14 AM
    Friday, December 14, 2012 3:19 PM

All replies

  • Check following

    - File Server feature is installed on all cluster nodes.

    - SMB server is enabled on all cluster nodes, and in particular server service is running on all the nodes

    - Verify that your firewall is not blocking SMB outgoing or incomming traffic. Make sure that rules would not block it for any network/IP.

    - Check if from one machine you can access \\<machine name>\c$ on the other machine.


    Regards, Vladimir Petter, Microsoft Corporation

    • Marked as answer by Lawrence,Lu Friday, December 14, 2012 9:23 AM
    Wednesday, December 5, 2012 5:49 PM
  • Hi Vladimir,

    Thanks for taking the time to respond. I've verified all of the criteria you've mentioned but still no luck!

    Wednesday, December 5, 2012 7:08 PM
  • Can you try following:

    1. Create a cluster without validation. Make sure that all nodes are up.

    2. Test that SMB over netft works

    2.1 On one of the cluster nodes run ipconfig /all and find IPv6 address on the "Microsoft Failover Cluster Virtual Adapter". It will look something like fe80::101a:e414:180d:2083%17. Let's call this node 1.

    2.2 Do the same on the other node so you will get address fe80::e89f:9699:d4d9:b16b%15. Let's call this node 2.

    3.3 Now on the Node 2 run "dir \\fe80::101a:e414:180d:2083%15\c$. I have highlighter above that IP should come from the Node 1 and the number after % should come from the Node 2. THe number after % might be the same on both nodes. This number tells Node 2 what interface to use when it goes out of the machine. If you do not provide this then it might not work since this is link local address

    3.4 Repeat steps 3.2 and 3.3 but this time try to reach Node 2 from Node 1. In my case I would use name “dir \\fe80::e89f:9699:d4d9:b16b%17\c$

    4. Test that NTLM authentication works

    4.1 On the Node 1 create a use “MYTESTUSER” with a password

    4.2 On the Node 2 create the same user with the exact same password.

    4.3 On Node 1 from an elevated command prompt using “runas /user:<Node 1 machine name>\MYUSER cmd.exe”. Go to the new command prompt verify that you are myuser using command whoami.

    4.4 Repeat test 3 from the command prompt created in the step 4.3. This time SMB will have to use NTLM and authenticate using MYTESTUSER.

    4.5 Go to the machine Node 2 and start command prompt there as described in 4.3

    4.6 Run test 3 from the Node 2 using command prompt created in step 4.5

    Let me know if any of these steps failed and what is the failure.


    Regards, Vladimir Petter, Microsoft Corporation

    • Marked as answer by Lawrence,Lu Friday, December 14, 2012 9:23 AM
    • Unmarked as answer by robdtec Friday, December 14, 2012 3:20 PM
    Wednesday, December 5, 2012 7:39 PM
  • Thanks again, Vladimir.

    It is stopping at step 4.6. It works from node 1 to node 2 but not in reverse.

    1 to 2:

    C:\Windows\system32>dir \\fe80::75fb:764d:ccd6:3f2d%21\c$
     Volume in drive \\fe80::75fb:764d:ccd6:3f2d%21\c$ has no label.
     Volume Serial Number is E228-2C64

     Directory of \\fe80::75fb:764d:ccd6:3f2d%21\c$

    12/05/2012  02:08 PM    <DIR>          ClusterStorage
    11/30/2012  01:03 PM    <DIR>          PerfLogs
    12/04/2012  03:25 PM    <DIR>          Program Files
    12/04/2012  03:25 PM    <DIR>          Program Files (x86)
    11/30/2012  11:08 AM    <DIR>          Users
    11/30/2012  02:12 PM    <DIR>          win7 x64-1
    11/30/2012  11:44 AM    <DIR>          Windows
                   0 File(s)              0 bytes
                   7 Dir(s)  869,506,834,432 bytes free

    2 to 1:

    C:\Windows\system32>dir \\fe80::54ef:2bdc:96a7:d94f%21\c$
    Access is denied.

    Wednesday, December 5, 2012 8:28 PM
  • Is interface id (the number after %) on the NetFt adapter identical on both machines? I see you are using %21 in both cases. Note that when trying from 1 to 2 (step 3.4) I used interface # from the node 1.

    If this is the issue then can you check if you are also getting access denied when you are trying to get from 2 to 1 using 1's machine name or IPs (v4 or v6) on the physical NICs?


    Regards, Vladimir Petter, Microsoft Corporation

    Wednesday, December 5, 2012 8:38 PM
  • I'm sorry, I misspoke. Its actually denied on both. I had accidentally run the command from 1->2 under an existing administrative account instead of the MYTESTUSER account.

    So, in fact, using those two general user accounts (duplicated on each machine), I get Access is denied. And to your other question, yes the interface ID is the same.

    Wednesday, December 5, 2012 8:43 PM
  • Are you also getting access denied when you are trying to access 1->2 or 2->1 using computer name or IP addresses on the physical NICs?

    I am wondering if you have a policy pushed by your AD to the cluster nodes that disables NTLM completely or some aspects of NTLM.


    Regards, Vladimir Petter, Microsoft Corporation

    Thursday, December 6, 2012 3:02 AM
  • If using the comp name/IP its being blocked when trying to access using the MYTESTUSER account, which is a normal user account. When using a domain admin account it works.
    Thursday, December 6, 2012 1:51 PM
  • For Cluster Shared Volume cluster relies on the NTLM feature where if the same user with the same password exists on two machines (Node1\User1 and Node2\User1) then if you logon on one machine using this user (log on on Node1 as Node1\user1), and then running as that user you logon to the other machine then you will be the same user on the other node (you will be Node2\User1 on the Node2). Note again that user must have the same password on both machines. 

    Please review with your AD administrator NTLM related policies and set them such that on the cluster nodes the mechanism described above would work.

    I am not an NTLM configuration expert so I cannot point you to exact setting that causes that, but hopefully one of these links will give you a direction

    http://technet.microsoft.com/en-us/library/cc773388(v=ws.10).aspx#BKMK_DisableLANManager

    http://technet.microsoft.com/en-us/library/bb877987.aspx

    Once you pin down the setting that caused the trouble please do let me know. I am interested in taking it back to the product team so in the next version validation tool can tell you not only what is not working, but hopefully would also suggest how to fix it.

    Regards, Vladimir Petter, Microsoft Corporation



    Thursday, December 6, 2012 4:27 PM
  • Vladimir,

    In order to do a quick test of this, I had the AD objects moved to an OU without any policies applied. The problem did not resolve itself.

    I did notice, though, that during the validation tests the wizard created a user "CliTest2" that had no group membership.

    I ended up destroying the cluster and starting over. The cluster validation wizard came back fine before creating. After creating the cluster it then had the same warning. Hmm.

    • Edited by robdtec Friday, December 7, 2012 4:17 PM
    Friday, December 7, 2012 2:24 PM
  • That is correct, cluster validation creates a local user that is not member of any groups.

    I cannot think of any reason why cluster would make any difference, except that some tests might not be possible and would not run without cluster being created. For instance without created cluster there is not virtual network on top of netft virtual miniport so we most likely are skipping this test.

    I am not sure if moving computer in AD would revert back the policies that are already applied to the machine.

    I would suggest to first get NTLM authentication to work using computer names and physical NICs. Most likely once this issue is resolved it will take care of authentication over netft virtual routes.

    If you have audit events enabled then check out Security Event log (run eventvwr) on the client and server after runing the repro. Perhaps there will be some events telling about failed authentication using MYTESTUSER, and description or status code that might hit what is failing.

    Also after running a repro try to look at SMB event channels. In the Event Viewer enable "Show Analytic and Debug Logs"

    and then look for these channels

    Operational channels are already enabled

    Analytic and Diagnostic channels you would need to enable, run repro, disable and then you will be able to see the events.

    Additional information about these channels can be found here

    http://social.technet.microsoft.com/Search/en-US?query=Microsoft-Windows-SMBClient&ac=8

    http://social.technet.microsoft.com/Search/en-US?query=Microsoft-Windows-SmbServer&ac=8


    Regards, Vladimir Petter, Microsoft Corporation

    Friday, December 7, 2012 6:02 PM
  • Hi,

    I would like to confirm what is the current situation? Have you resolved the problem or do you have any further progress?

    If there is anything that we can do for you, please do not hesitate to let us know, and we will be happy to help.


    Lawrence

    TechNet Community Support

    Monday, December 10, 2012 2:14 AM
  • Hi Lawrence,

    I'm not really having any luck.  The security event log shows this during an attempted access:

    Audit Success 12/10/2012 7:56:55 AM Microsoft Windows security auditing. 4634 Logoff
    Audit Failure 12/10/2012 7:56:43 AM Microsoft Windows security auditing. 5140 File Share
    Audit Failure 12/10/2012 7:56:43 AM Microsoft Windows security auditing. 5140 File Share
    Audit Success 12/10/2012 7:56:43 AM Microsoft Windows security auditing. 5140 File Share
    Audit Success 12/10/2012 7:56:43 AM Microsoft Windows security auditing. 4624 Logon
    Audit Success 12/10/2012 7:56:43 AM Microsoft Windows security auditing. 4776 Credential Validation

    But the details don't provide me with anything I didn't know (who, where, to where). I enabled the analytic and diagnostic channels but the data that comes through isn't user-readable and I haven't found a way to see what its saying.

    Monday, December 10, 2012 2:37 PM
  • Try posting/searching the question on the "File Services and Storage" or "Security" forums

    http://social.technet.microsoft.com/Forums/en-US/winserversecurity/threads

    http://social.technet.microsoft.com/Forums/en-US/winserverfiles/threads


    Regards, Vladimir Petter, Microsoft Corporation

    Monday, December 10, 2012 7:05 PM
  • Hi,

    Could you recap your problem ? Building a cluster in a lab and It seams this is working for you ( post above )

    But I'm lost on what is your current problem. and what Storage are you using ?

    Building a windows 2012 cluster

    http://blogs.msdn.com/b/clustering/archive/2012/05/01/10299698.aspx

    Or see my blog

    http://robertsmit.wordpress.com/2012/03/20/how-to-create-a-windows-server-8-cluster-windows-server-2012-cluster/


    Greetings, Robert Smit [MVP] http://robertsmit.wordpress.com/ “Please click "Vote As Helpful" if it is helpful for you and Proposed As Answer”

    Tuesday, December 11, 2012 7:01 PM
  • Basically I can create the cluster but whenever I go to use the cluster storage it is giving me an access denied error. Specifically, I'm wanting to use this for storing hyper-v VMs but I can't even move a VM into the CSV. When I run the validation on the cluster (before creation) everything comes up fine. After I created the cluster it errors on "Validate CSV settings" (see above).

    I'm using a Sun Storage J4200 for the storage array with iscsi. And you correctly mentioned that this is just a lab environment.

    I don't quite get what Vladimir was getting at. I rebuilt the two host machines and without having them in a domain I still can't do what he thinks it should be able to do when the accounts are standard local users. More "access denied" errors.

    Tuesday, December 11, 2012 8:42 PM
  • In 2012 cluster requires a domain. You cannot create a cluster without joining nodes to a domain.

    For CSV to work you also need following feature to work

    <start quote>

    For Cluster Shared Volume cluster relies on the NTLM feature where if the same user with the same password exists on two machines (Node1\User1 and Node2\User1) then if you logon on one machine using this user (log on on Node1 as Node1\user1), and then running as that user you logon to the other machine then you will be the same user on the other node (you will be Node2\User1 on the Node2). Note again that user must have the same password on both machines.

    <end quote>

    Cluster validation test is complaining that it does not. Your tests above also confirm that it fails with access-denied.

    My last suggestion was to try asking on the "File Services and Storage" or "Security" forums why it might be failing.


    Regards, Vladimir Petter, Microsoft Corporation

    Tuesday, December 11, 2012 9:32 PM
  • Hi,

    Just what vladimir said.  You can't build a cluster without a DC, you need a DC. and I should be the same domain. but if you do have a dc, Is the Cluster netbios name created in the Active directory ? if yes the cluster service is running on both nodes ?

    Are you using a quorum/wittness disk ?

    But you said I have a cluster, How did you create the cluster ? , just enabling the Role is not creating the cluster. and how is your network configuration ?

    One nic for the Lan , one for ISCSI , one for CSV , One for Cluster HB.  and all have his own ip subnet.

    And more important is that your iscsi lun need access to both nodes

    Please read the blog post : below is everything what you need to build a cluster and to get started with your csv.

    Building a windows 2012 cluster

    http://blogs.msdn.com/b/clustering/archive/2012/05/01/10299698.aspx


    Greetings, Robert Smit [MVP] http://robertsmit.wordpress.com/ “Please click "Vote As Helpful" if it is helpful for you and Proposed As Answer”

    Wednesday, December 12, 2012 7:29 AM
  • I said I rebuilt the machines to test out Vladimir's NTLM suggestion without any AD policies getting in the way. Two fresh 2012 OS installs didn't do what Vladimir said they should. Other than that, of course the machines were in AD for the cluster. The links you're giving me are really basic stuff. Kind of regressing in quality of suggestions... not that I don't appreciate the attempt.

    Same domain? yes

    Cluster name in AD? yes

    Service running? yes

    witness disk? yes

    network setup? 1 vlan iscsi, 1 vlan live migration/cluster heartbeat, 1 nic for host access, 1 nic for VM trunk (i only have 4 nics on these test machines)

    iscsi targets? accessible by both nodes

    • Marked as answer by Lawrence,Lu Friday, December 14, 2012 9:22 AM
    • Unmarked as answer by robdtec Friday, December 14, 2012 3:20 PM
    Wednesday, December 12, 2012 1:23 PM
  • Hi,

    With this configuration you should be able to use CSV with out any troubles.

    but remember never share something on the node for cluster usage. always do this in a cluster resource.


    Greetings, Robert Smit [MVP] http://robertsmit.wordpress.com/ “Please click "Vote As Helpful" if it is helpful for you and Proposed As Answer”

    Wednesday, December 12, 2012 3:42 PM
  • I rebuilt the whole thing identically as before and the validation error went away. I can get a little bit further but I'm having issues with the CSV itself now. I'll ask that in another question. If I figure out what the issue for this question was, I'll post it.
    • Edited by robdtec Wednesday, December 12, 2012 4:37 PM
    • Marked as answer by Lawrence,Lu Friday, December 14, 2012 9:22 AM
    • Unmarked as answer by robdtec Friday, December 14, 2012 3:19 PM
    Wednesday, December 12, 2012 4:37 PM
  • I went line by line through all of our settings. It turns out someone higher in the AD forest was pushing down a registry change through group policy.

    They were disabling SMBv2 using

    Hive HKEY_LOCAL_MACHINE
    Key path SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
    Value name smb2
    Value type REG_DWORD
    Value data 0x0 (0)
    • Proposed as answer by uniken1 Tuesday, January 15, 2013 11:25 AM
    • Marked as answer by robdtec Thursday, January 31, 2013 1:14 AM
    Friday, December 14, 2012 3:19 PM
  • I try to solve same problem for some weeks.
    Set smb2=dword:0x00000001 fix my problem, Thanks man.

    Tuesday, January 15, 2013 11:06 AM
  • Cluster Shared Volumes (CSV) in Windows Server 2012 has a dependency on SMB 3.0.  By setting that registry key it disables SMB2 or higher, forcing it to SMB1 and breaking CSV.

    Just curious, why did you disable SMB2?? 


    Tuesday, January 15, 2013 3:43 PM
  • I dont do this myself. I dont know why, but Citrix Provisioning Services disable it when installing.
    Wednesday, January 16, 2013 3:45 AM