locked
ATA Lightweight Gateway + VMWare = Dropped port mirrored network traffic Alerts RRS feed

  • Question

  • We recently implemented ATA with the Lightweight Gateway model across about a dozen Domain Controllers this past weekend. Half of the DCs are physical and the other half are VMWare VMs. We sized both the ATA Center and the Domain Controllers based off the packet per/sec recommendations and took the average. For reasons unknown, all of our VMs are alerting with the "Dropped port mirrored network traffic" alert across all of the VMs, but not one of the physicals has alerted.

    Has anyone else experienced this behavior and resolved it successfully? It appears to be a false-positive, but I would hate to hit the "ignore me" button and not see it when valid.

    Thanks!

    -Chris



    • Edited by SRVRGEEK Wednesday, July 13, 2016 1:50 PM typo
    Tuesday, July 12, 2016 5:16 PM

Answers

  • Hello,

    We find the following method from:

    https://info.pleasantsolutions.com/blog/vmware_networking_speed_issue

    On Windows, go to 

    Network Connection icon   -> Properties -> Configure -> Advanced Tab.

    Look for   "TsoEnable", "LargeSendOffload", "IPv4   TSO Offload" or otherwise and set it to 0 / Disabled.

    You may also want to look for things like "IPv4 Giant TSO Offload", I turn them off to be safe but I haven't actually confirmed if they matter or not. Look over and check for any other Tso related settings too.

    This seems to resolve the issue at several customers.

    Thanks,

     Microsoft ATA Team


    Monday, September 5, 2016 4:18 PM

All replies

  • Hi,

    we had to give the GWs more resources. That resolved this problem in our systems.

    Regards

    Thursday, July 14, 2016 12:42 PM
  • We're seeing this too - but only on 2012R2 VM DCs, not 2008R2 VM DCs which leads us to suspect its a VMware/2012R2 false positive.
    Friday, July 15, 2016 1:43 PM
  • Thanks for the reply. We have gone over the resource requirements and even moved a VM to a dedicated host, but there was no change in the alert behavior.

    -Chris

    Friday, July 15, 2016 3:18 PM
  • Hi Jonathan,

    We are checking if this issue (2012 R2 Virtual DCs) may related to RSS being disabled on VMXNET3 NICs.

    Any chance you can enable RSS on the NIC and see if this fix the issue?

    More information here:

    https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2008925

    Thanks,

     Microsoft ATA Team

    Thursday, July 28, 2016 6:35 AM
  • ^ not working for us. Also 2012 R2 VMware DCs with RSS enabled on VMXNET3 Nic's.
    Wednesday, August 3, 2016 6:57 AM
  • Hi

    I've applied this (RSS enabled) to two of our more 'vocal' servers but this hasn't made any difference so far (unless a reboot is needed).

    I'll check the other two options mentioned in the article

    Kind regards

    Jonathan

    Friday, August 5, 2016 9:47 AM
  • Hi Jonathan (and others),

    If RSS change did not solve the issue, can you check the following applicable to your environment:

    https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2129176

    If it does - can you try to disable RSC and see if this make things better?

    Thanks,

        Microsoft ATA Team.

    Wednesday, August 10, 2016 7:05 AM
  • Hi,

    this also doesn't work (We are using VMware 5.5 though). Still getting dropped traffic alerts.

    Regards

    Friday, August 12, 2016 5:40 AM
  • we're already on 6.0 Update 2 but I've applied anyway. in hope more than anything else :D
    Friday, August 12, 2016 10:43 AM
  • No joy (as expected really as already on 6.0 Update2)
    Thursday, August 18, 2016 6:58 AM
  • Hello,

    We find the following method from:

    https://info.pleasantsolutions.com/blog/vmware_networking_speed_issue

    On Windows, go to 

    Network Connection icon   -> Properties -> Configure -> Advanced Tab.

    Look for   "TsoEnable", "LargeSendOffload", "IPv4   TSO Offload" or otherwise and set it to 0 / Disabled.

    You may also want to look for things like "IPv4 Giant TSO Offload", I turn them off to be safe but I haven't actually confirmed if they matter or not. Look over and check for any other Tso related settings too.

    This seems to resolve the issue at several customers.

    Thanks,

     Microsoft ATA Team


    Monday, September 5, 2016 4:18 PM
  • Thank you Ophir and the Microsoft ATA R&D team! After many attempts to narrow down the issue, the TSO change fixed the issue for us. We have not had any alerts and/or dropped events since implementing the change.

    Thank you for your tireless help!

    -Chris

    Tuesday, September 6, 2016 1:57 PM
  • Thank you Ophir and the Microsoft ATA R&D team! After many attempts to narrow down the issue, the TSO change fixed the issue for us. We have not had any alerts and/or dropped events since implementing the change.

    Thank you for your tireless help!

    -Chris

    Hello Chris,

    Do you recall which specific 'TSO change' fixed this issue?

    - Disabling 'IPv4 TSO Offload'?

    - Disabling 'Large Send Offload V2 (IPv4)?

    or what?

    Kind regards,

    Jos


    • Edited by le_phreak Thursday, November 1, 2018 2:10 PM
    Thursday, November 1, 2018 2:09 PM