locked
Weird Hyper-V Network Issue RRS feed

  • Question

  • Hi,

    We're having a very odd and elusive networking problem with a Hyper-V 2012 R2 server.

    Shortly are upgrading all our servers to Server 2012 R2 and migrating to Hyper-V 2012 R2 users started to notice that Outlook would disconnect and redirected documents would become inaccessible randomly at varying times throughout the day.

    After some troubleshooting we managed to isolate the following behaviour.

    Pings to the VM's drop off first followed shortly by pings to the Hyper-V host.

    The packets are dropped for a period of around 2 minutes intermittently around 1-2 times per day.

    The timing seems completely random.

    Neither the VM's nor Hyper-V host register any loss of network connection.

    VM's can not talk to each other over the vSwitch whilst the problem is occurring.

    Normal service resumes after usually around 2 minutes or so.

    Nothing seems to be logged in the event log on any of the machines to do with network issues. The only trace we can find is EventID 2070 been logged on the exchange servers application log to say that the DC is unavailable. And on the domain controller/file server we can see LDAP session time outs and SMB handles been scavenged.

    The server is running on a ML350P Gen8  (Broadcom NIC) - All firmware and drivers have been updated.

    All servers are up to date with all the latest updates.

    We have also tried a second Intel based NIC but this has made no difference as did swapping all network cables and the switch.

    There are currently 4 VM's connected to a single vSwitch which is linked to a dedicated port on the Intel NIC - management is not allowed through this interface but is done through the second interface on the Intel NIC.

    Has anyone come across this before? It is causing a lot of problems with the client and we are very close to rebuilding the Hyper-V host with an earlier version of Windows server.

    Thanks,

    Richard.

     

      

      .

    Tuesday, June 10, 2014 12:57 PM

Answers

  • Hi Gil,

    Thanks for the reply.

    No NIC teaming is not enabled. Its a dual port NIC with one port for management and one for the vSwitch.

    I've disable VMQ on the NIC using Powershell this afternoon so will wait and see if this has any effect....

    • Marked as answer by Elton_Ji Wednesday, June 18, 2014 4:03 PM
    Tuesday, June 10, 2014 8:18 PM
  • Well after coming very close to migrating to a new vmhost it looks like disabling VMQ on all nics has resolved the issues. How and why I do not know as it is supposedly supported on both the in built Broadcom nic and the additional Intel nic we installed.

    I ran the following powershell command to disable it

    Disable-NetAdapterVMQ

    I then used Get-NetAdapterVMQ to confirm.

    After doing this it re-occurred once within 24 hours. Since then it has not happened for 5 days and counting.





    • Edited by xdmon Monday, June 16, 2014 9:33 AM
    • Marked as answer by xdmon Monday, June 16, 2014 9:33 AM
    Monday, June 16, 2014 9:31 AM

All replies

  • Hi,

    Do you have Nic teaming enabled and if so it it native windows or driver based?

    I've seen issues like this with faulty LACP settings or switches that for some reason do not support dynamic nic teaming.


    Gil Gross | Technical Lead | G-Net Network Solutions | www.g-net.co.il | plz visit my blog - gilgrossblog.wordpress.com

    Tuesday, June 10, 2014 4:42 PM
  • Hi Gil,

    Thanks for the reply.

    No NIC teaming is not enabled. Its a dual port NIC with one port for management and one for the vSwitch.

    I've disable VMQ on the NIC using Powershell this afternoon so will wait and see if this has any effect....

    • Marked as answer by Elton_Ji Wednesday, June 18, 2014 4:03 PM
    Tuesday, June 10, 2014 8:18 PM
  • Well after coming very close to migrating to a new vmhost it looks like disabling VMQ on all nics has resolved the issues. How and why I do not know as it is supposedly supported on both the in built Broadcom nic and the additional Intel nic we installed.

    I ran the following powershell command to disable it

    Disable-NetAdapterVMQ

    I then used Get-NetAdapterVMQ to confirm.

    After doing this it re-occurred once within 24 hours. Since then it has not happened for 5 days and counting.





    • Edited by xdmon Monday, June 16, 2014 9:33 AM
    • Marked as answer by xdmon Monday, June 16, 2014 9:33 AM
    Monday, June 16, 2014 9:31 AM