locked
S2D migration 2016 -> 2019 - RDMA Problems RRS feed

  • Question

  • Hi..

    we have a two node Hyper-V S2D 2016 Cluster of Dell R730 and Mellanox dual port ConnectX-3 10GbE for handle the East/West traffic. The cards in both nodes are directly connected by DAC Cables.

    The S2D 2016 Cluster working fine with this configuration. East/West traffic using RDMA.

    So after upgrading one of the Nodes to 2019, all the traffic during the Mellanox cards are no longer processed by RDMA (RDMA Performance Counter are emtpy until "RDMA Failed Connection Attempts" which getting higher...)

    No RDMA support results in a bad performance and high CPU load :(

    I'm using this HowTo to set all settings on the new node: https://docs.microsoft.com/de-de/windows-server/virtualization/hyper-v-virtual-switch/rdma-and-switch-embedded-teaming

    I'm also compared any related powershell output of the 2016 and 2019 node.

    Is this a by design behavior, that RDMA not working in a mixed OS environment?

    I would like to wait with the upgrade of the second node until the question is resolved...

    Please help :)

    Alex






    Friday, October 25, 2019 9:39 AM

Answers

  • I solved it by myself:

    It seems that the Win 2019 Mellanox NIC driver was not compatible with the 2 years old 2016 one on the other node.

    So I updated also the NIC driver on the 2016 node to the latest version and make sure that both nodes using RoCE Mode 1.0:

    Set-MlnxDriverCoreSetting -RoceMode 1.0 -Confirm:$false

    Monday, October 28, 2019 7:44 AM

All replies

  • I solved it by myself:

    It seems that the Win 2019 Mellanox NIC driver was not compatible with the 2 years old 2016 one on the other node.

    So I updated also the NIC driver on the 2016 node to the latest version and make sure that both nodes using RoCE Mode 1.0:

    Set-MlnxDriverCoreSetting -RoceMode 1.0 -Confirm:$false

    Monday, October 28, 2019 7:44 AM
  • Hi,

    I'm very glad that your issue was resolved successfully!

    Thanks for your sharing and support.

    Have a nice day!

    Best regards,

    Michael


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com

    Monday, October 28, 2019 9:32 AM