none
SMB 3.0 with Infiniband / RDMA RRS feed

All replies

  • Did you update the adapter drivers?
    Tuesday, October 30, 2012 6:15 PM
  • Hi Sean,

    Yes I did. Grabbed them from the link in the top of the blog post.

    Thanks,

    Ryan

    Tuesday, October 30, 2012 6:21 PM
  • Hi,

    Thank you for your question.
    I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

    Thank you for your understanding and support.

    Best Regards,
    Aiden


    Aiden Cao

    TechNet Community Support

    Thursday, November 1, 2012 6:27 AM
    Moderator
  • Thanks Aiden.
    Thursday, November 1, 2012 2:23 PM
  • The 2.09.1000 version of Mellanox firmware is old and does not support RDMA.  You should get an update from Mellanox's website.

    Thursday, November 1, 2012 7:26 PM
  • Hi Jeffrey,

    Thank you for the reply. According to their website (http://mellanox.com/content/pages.php?pg=firmware_table_ConnectX2IB), 2.09.1000 is what is currently available. Additionally, shouldn't the WinOF VPI installer identify an older firmware and update that?

    Thanks,

    Ryan

    Thursday, November 1, 2012 7:30 PM
  • Hi,

    Here are some information for your reference, see if it can help you:

    RDMA is enabled by default and there is typically no need to disable it. To disable RDMA, you can use the following Windows PowerShell cmdlets:
    To disable RDMA for a specific interface:
    Disable-NetAdapterRdma <name>

    To disable RDMA for all interfaces:
    Set-NetOffloadGlobalSetting -NetworkDirect Disabled

    You can re-enable RDMA with the following:
    To re-enable RDMA for a specific interface:
    Enable-NetAdapterRDMA <name>

    To re-enable RDMA for all interfaces:
    Set-NetOffloadGlobalSetting -NetworkDirect Enabled

    Thanks.

    Best Regards,

    Annie Gu

    Friday, November 2, 2012 5:46 AM
  • Hi Annie,

    Thank you for the reply -- I believe that is all set correctly; correct me if I'm wrong.

    Friday, November 2, 2012 1:41 PM
  • Hi,

    I think it is correct, and.. would you like to try re-enable the RDMA on the server again for a test:

    To re-enable RDMA for all interfaces:

    Set-NetOffloadGlobalSetting -NetworkDirect Enabled

    After that, please try checking if they are all enabled. If not, I think there can be something wrong on the hardware layer but not the OS layer.

    I'm trying to provide more information for your reference:


    Note: A team of RDMA-capable NICs is always reported as non-RDMA capable. If you intend to use the RDMA capabilities of the NIC, do not team them. Please check if you teamed these NIC cards.

    Then, here is some general troubleshooting methods:


    Troubleshooting SMB Multichannel
    ===================================
    Use the following tools and methodolgy to troubleshoot SMB Multichannel operation.
    Verifying if you are using SMB Multichannel

    You can use the following steps to verify you are using SMB Multichannel.

    Step 1: Verify network adapter configuration
    Use the following Windows PowerShell cmdlets to verify you have multiple NICs and/or to verify the RSS and RDMA capabilities of the NICs. Run on both the SMB server and the SMB client.
    Get-NetAdapter
    Get-NetAdapterRSS
    Get-NetAdapterRDMA

    Step 2: Verify SMB configuration
    Use the following Windows PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RSS and RDMA capabilities are being properly identified.
    On the SMB client, run the following Windows PowerShell cmdlets:
    Get-SmbClientConfiguration | Select EnableMultichannel
    Get-SmbClientNetworkInterface
    On the SMB server, run the following Windows PowerShell cmdlets:
    Get-SmbServerConfiguration | Select EnableMultichannel
    Get-SmbServerNetworkInterface

    Step 3: Verify the SMB connection
    On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a Windows PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect (SMB 3.0) and that SMB Multichannel is working:
    Get-SmbConnection
    Get-SmbMultichannelConnection

    View SMB Multichannel Events
    SMB Multichannel events are recorded in the SMB Client log. You can look at this log using the Event Viewer or query it using Windows PowerShell. 

    Option 1: Using Event Viewer
    To view the SMB Multichannel events using Event Viewer, use the following steps on the SMB Client:
    Open Server Manager
    In Server Manager, click Tools, and then click Event Viewer
    In Event Viewer, expand the console tree to show Applications and Service Logs, Microsoft, Windows, SMB Client, Operational
    Click on Filter Current Log… in the Actions pane on the right and enter 30700-30705 on the filter for Event IDs.
    To view only errors:
    Click on Filter Current Log… in the Actions pane on the right and select the checkbox labeled Errors.

    Option 2: Using Windows PowerShell
    To view the SMB Multichannel events using Windows PowerShell, use the following cmdlet on the SMB client:
    Get-WinEvent -LogName Microsoft-Windows-SMBClient/Operational | 
    ? { $_.Id -ge 30700 –and $_.Id –le 30705 } 
    To list only errors, use the following cmdlet:
    Get-WinEvent -LogName Microsoft-Windows-SMBClient/Operational | 
    ? { $_.Id -ge 30700 –and $_.Id –le 30705 –and $_.Level –eq 2 } 

    View SMB Performance Counters
    There are several performance counters related to SMB in Windows Server 2012. To view SMB-related performance information, follow these steps:
    Open Server Manager
    In the Tools menu, open Performance Monitor
    In Performance Monitoring, click on Performance Monitor in the console tree.
    Switch to the Report View by pressing CTRL-G twice (or using the icon on the toolbar)
    Add performance counters to the view by pressing CTRL-N (or using the icon on the toolbar)

    The following Performance Counters are useful when looking at activities related to SMB, SMB Direct and Network Direct:
    Counter Name Shows information for
    SMB2 Server Shares Shares on the SMB Server
    SMB2 Server Sessions Sessions on the SMB Server
    SMB2 Client Shares Shares on the SMB Client 


    Monday, November 5, 2012 9:31 AM
  • From the last output, it shows you're running firmware version 2.09.1000, which is an older version.
    You should review this blog post: http://blogs.technet.com/b/josebda/archive/2013/01/18/minimum-version-of-mellanox-firmware-required-for-running-smb-direct-in-windows-server-2012.aspx

    Jose Barreto


    Jose Barreto

    Friday, March 1, 2013 1:35 AM
  • We are using the Dell MCQH29-XDR and as shown here the 2.09.1000 is the most current firmware. 

    Mellanox Dell Cards & Firmware

    So is there no way to get RDMA working with SMB?

    Thursday, March 21, 2013 4:04 PM