locked
S2D RDMA Validation RRS feed

  • Question

  • Good day all.

    I'm building a new Hyper-Converged stack consisting of 3 servers with a mix of SSD and spinning disk.

    During validation, I am attempting to run the Test-RDMA.ps1 script referenced here:
    https://docs.microsoft.com/en-us/windows-server/networking/technologies/conv-nic/cnic-app-troubleshoot

    I don't get far before it throws an error stating that RDMA is not enabled on the vEthernet adapter. However, I've enabled RDMA and running get-NetAdapterRDMA returns a value of true. 

    Looking for troubleshooting tips, I came across the article referenced above. One of the validation steps is to run 

    Get-SmbClientNetworkInterface

    When I run this, I get a different result for RDMA Capable. It is returning FALSE for the adapter. 

    My question is: how do I enable this value for this? There doesn't seem to be a corresponding 'set' command to go with this.

    Thanks!

    Monday, March 23, 2020 11:16 PM

Answers

  • Actually, coming back around to this, there were more issues at play.

    1. While I had set the SMB NICs to RDMA capable, the VNICS were not. This also has to be done.

    2. This did fix the setting but we were still having issues with testing. Turns out that jumbo packets were not turned on on the Mellanox switch. Enabling this along with the above steps fully rectified the situation.

    Happy computing! 


    Thursday, March 26, 2020 4:12 PM

All replies

  • Hi ,

    Make sure Network Direct (RDMA) is enabled on both the physical NIC and host vNIC.

    Please run the command of Enable-NetAdapterRdma or Set-NetAdapterRdma to enable RDMA again:

    Set-NetAdapterRdma -Name "MyAdapter" -Enabled $True
    Enable-NetAdapterRdma -Name "MyAdapter"

    For your reference:

    Set-NetAdapterRdma

    Enable-NetAdapterRdma

    If it still doesn't work, please post the screenshots of the command of get-NetAdapterRDMA and Get-SmbClientNetworkInterface.

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Tuesday, March 24, 2020 2:57 AM
  • Thanks, Candy. These commands have already been issued to the nodes.

    Turns out, there was a BIOS setting that was preventing the full RDMA access from working. After setting the on board controller to EFI mode instead of the default LEGACY, everything checks out.

    Tuesday, March 24, 2020 2:50 PM
  • Hi ,

    I am glad to hear that your issue was successfully resolved.

    In addition, thanks for sharing your solution in the forum as it would be helpful to anyone who encounters similar issues.

    If there is anything else we can do for you, please feel free to post in the forum.

    You could mark the reply as answer to close this thread.

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Wednesday, March 25, 2020 1:41 AM
  • Actually, coming back around to this, there were more issues at play.

    1. While I had set the SMB NICs to RDMA capable, the VNICS were not. This also has to be done.

    2. This did fix the setting but we were still having issues with testing. Turns out that jumbo packets were not turned on on the Mellanox switch. Enabling this along with the above steps fully rectified the situation.

    Happy computing! 


    Thursday, March 26, 2020 4:12 PM
  • Hi ,

    I really appreciate your ability to troubleshoot, analyze the issue and find the correct solutions. 

    Have a nice day!

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Friday, March 27, 2020 2:51 AM