none
2012 Hyper-V host crashes

    Question

  • we are getting some random crashes occasionally.  debug always has the same results.  can anyone make anything of this?  the host is up to date with both drivers and updates.  I've been all over the host and can't find any issues.   thank you

    Loading Dump File [C:\Minidump\092613-10374-01.dmp]
    Mini Kernel Dump File: Only registers and stack trace are available

    Symbol search path is: SRV*C:\symbolfiles*http://msdl.microsoft.com/download/symbols;C:\Symbols
    Executable search path is:
    Windows 7 Kernel Version 9200 MP (16 procs) Free x64
    Product: Server, suite: TerminalServer DataCenter SingleUserTS
    Built by: 9200.16628.amd64fre.win8_gdr.130531-1504
    Machine Name:
    Kernel base = 0xfffff800`b526d000 PsLoadedModuleList = 0xfffff800`b5539a20
    Debug session time: Thu Sep 26 02:07:18.299 2013 (UTC - 7:00)
    System Uptime: 18 days 8:02:55.169
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ...............
    Loading User Symbols
    Loading unloaded module list
    ......
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {3, 2, 0, fffff880048f77a0}

    Probably caused by : vmswitch.sys ( vmswitch!memmove+1e0 )

    Followup: MachineOwner
    ---------

    0: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 0000000000000003, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
    Arg4: fffff880048f77a0, address which referenced memory

    Debugging Details:
    ------------------


    READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800b55c5168
    unable to get nt!MmPoolCodeStart
    unable to get nt!MmPoolCodeEnd
     0000000000000003

    CURRENT_IRQL:  2

    FAULTING_IP:
    vmswitch!memmove+1e0
    fffff880`048f77a0 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    CUSTOMER_CRASH_COUNT:  1

    DEFAULT_BUCKET_ID:  DRIVER_FAULT_SERVER_MINIDUMP

    BUGCHECK_STR:  0xD1

    PROCESS_NAME:  vmms.exe

    TRAP_FRAME:  fffff880082ce2a0 -- (.trap 0xfffff880082ce2a0)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=fffffa8049315dc8
    rdx=0000057fb6cea23c rsi=0000000000000000 rdi=0000000000000000
    rip=fffff880048f77a0 rsp=fffff880082ce438 rbp=fffff880082ce7a0
     r8=0000000000000004  r9=0000000000000000 r10=0000000000000024
    r11=fffffa8049315dc4 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    vmswitch!memmove+0x1e0:
    fffff880`048f77a0 8a440aff        mov     al,byte ptr [rdx+rcx-1] ds:e080:00000000`00000003=??
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff800b52c6769 to fffff800b52c7440

    STACK_TEXT: 
    fffff880`082ce158 fffff800`b52c6769 : 00000000`0000000a 00000000`00000003 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
    fffff880`082ce160 fffff800`b52c4fe0 : 00000000`00000000 00000000`0000001e fffff880`082ce300 fffff880`082ce2a0 : nt!KiBugCheckDispatch+0x69
    fffff880`082ce2a0 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x260


    STACK_COMMAND:  .bugcheck ; kb

    FOLLOWUP_IP:
    vmswitch!memmove+1e0
    fffff880`048f77a0 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    SYMBOL_NAME:  vmswitch!memmove+1e0

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: vmswitch

    IMAGE_NAME:  vmswitch.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  510cbec2

    FAILURE_BUCKET_ID:  X64_0xD1_vmswitch!memmove+1e0

    BUCKET_ID:  X64_0xD1_vmswitch!memmove+1e0

    Followup: MachineOwner


    BarrySDCA

    Thursday, September 26, 2013 6:46 PM

Answers

  • so in November I opened a ticket for this issue and Microsoft help was useless - absolutely embarrassing.    Well, it seems they finally published a patch:

    http://support.microsoft.com/kb/2913659/en-us

    Dated January 2014.  their patch is for only 2012 R2, which we are running now (after moving away from 2012 due to the same issue).

    If they had taken our ticket a little more seriously I'm sure they would have published the patch much sooner.

    I'm just glad we have one  - finally.

    thank you to everyone here who tried to assist.

    I should add that our boxes are up to date, but this vmswitch.sys driver does not seem to be included in the normal MS update.


    BarrySDCA


    • Marked as answer by BarrySDCA Tuesday, May 06, 2014 11:10 PM
    • Edited by BarrySDCA Tuesday, May 06, 2014 11:11 PM update vmswitch.sys info
    Tuesday, May 06, 2014 11:10 PM

All replies

  • You have a miss-behaving driver.

    Since you updated drivers - which ones?  And did you check if the corresponding firmware also needed to be updated?

    There are many cases where both need to happen together.  Especially with NIC drivers, and your smells like a NIC driver.

    Also, the first step of CSS is to tell you to remove any 3rd party / manufacturer drivers and try the Windows in-box drivers first and see if the problem persists.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.


    Thursday, September 26, 2013 6:57 PM
    Moderator
  • thank you for the fast reply Brian.

    the issue has been seldom, but persistent.  both before and after driver updates.  only two real drivers on the box, adaptec raid card and intel converged nic.  both have been updated before and after.  adaptec firmware has for sure.  I am not sure about the nic, but I will check.

    I can't remove the adaptec driver, windows does not recognize it otherwise.

    Intel driver may be a problem to remove too with these converged nics and VMQ's.  I'll check for a firmware update, but I don't think there is one.

    thank you


    BarrySDCA

    Thursday, September 26, 2013 7:33 PM
  • ya I checked, Intel NIC's do not have firmware to update.  the drivers update them.  so it's up to date and has been for months (updated as new ones are published).  still this problem persists

    BarrySDCA


    • Edited by BarrySDCA Thursday, September 26, 2013 9:39 PM
    Thursday, September 26, 2013 9:39 PM
  • Hi,

    The BSOD can caused by the damaged NIC adapter, If you have the more NIC adapter try to install the new one and monitor again.

    Thanks.


    Alex Lv

    Monday, September 30, 2013 6:20 AM
    Moderator
  • great theory Alex - thank you.   except, the problem has occurred on a couple different machines.  all with the same debug I posted at the top of this thread.

    anyone else?  I'm about to toss out all our Intel nic's if I can't find a solution.

    thank you


    BarrySDCA

    Monday, September 30, 2013 2:21 PM
  • more information about the issue:

    it seems the crash may occur when a stopped VM is booted.  not 100% sure but starting to look that way.  it's not a sure thing, difficult to reproduce

    the NIC's are setup in a team, via VMM 2012.


    BarrySDCA

    Tuesday, October 01, 2013 1:57 AM
  • could be related to this:

    http://www.aidanfinn.com/?p=14303


    BarrySDCA

    Wednesday, October 02, 2013 10:21 PM
  • that is not it...I disabled sr-iov (should not have been enabled to begin with) and we just had another crash.

    the hyper-v nic's are in a team setup by vmm 2012.  latest intel NIC driver too.  540-t2

    new dump:


    Microsoft (R) Windows Debugger Version 6.2.9200.16384 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\WORKING\crash_dump\hv23\MEMORY.DMP]
    Kernel Bitmap Dump File: Only kernel address space is available

    Symbol search path is: SRV*C:\symbolfiles*http://msdl.microsoft.com/download/symbols;C:\Symbols
    Executable search path is:
    Windows 8 Kernel Version 9200 MP (16 procs) Free x64
    Product: Server, suite: TerminalServer DataCenter SingleUserTS
    Built by: 9200.16628.amd64fre.win8_gdr.130531-1504
    Machine Name:
    Kernel base = 0xfffff800`a5c1c000 PsLoadedModuleList = 0xfffff800`a5ee8a20
    Debug session time: Sat Oct  5 07:43:41.818 2013 (UTC - 7:00)
    System Uptime: 5 days 21:40:09.239
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ..............
    Loading User Symbols
    PEB is paged out (Peb.Ldr = 000007f7`07ee3018).  Type ".hh dbgerr001" for details
    Loading unloaded module list
    .........
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {3, 2, 0, fffff8800472a7a0}

    Page fd4fd2 not present in the dump file. Type ".hh dbgerr004" for details
    Probably caused by : vmswitch.sys ( vmswitch!memcpy+1e0 )

    Followup: MachineOwner
    ---------

    2: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 0000000000000003, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
    Arg4: fffff8800472a7a0, address which referenced memory

    Debugging Details:
    ------------------

    Page fd4fd2 not present in the dump file. Type ".hh dbgerr004" for details

    READ_ADDRESS:  0000000000000003

    CURRENT_IRQL:  2

    FAULTING_IP:
    vmswitch!memcpy+1e0
    fffff880`0472a7a0 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

    BUGCHECK_STR:  AV

    PROCESS_NAME:  vmms.exe

    TRAP_FRAME:  fffff880099952a0 -- (.trap 0xfffff880099952a0)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=fffffa8055fabe20
    rdx=0000057faa0541e4 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff8800472a7a0 rsp=fffff88009995438 rbp=fffff880099957a0
     r8=0000000000000004  r9=0000000000000000 r10=0000000000000024
    r11=fffffa8055fabe1c r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    vmswitch!memcpy+0x1e0:
    fffff880`0472a7a0 8a440aff        mov     al,byte ptr [rdx+rcx-1] ds:00000000`00000003=??
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff800a5c75769 to fffff800a5c76440

    STACK_TEXT: 
    fffff880`09995158 fffff800`a5c75769 : 00000000`0000000a 00000000`00000003 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
    fffff880`09995160 fffff800`a5c73fe0 : 00000000`00000000 00000000`0000001b 00000000`00000000 fffff880`099952a0 : nt!KiBugCheckDispatch+0x69
    fffff880`099952a0 fffff880`0472a7a0 : fffff880`0471ef35 00000000`0000001b fffff880`099957a0 fffffa80`55faace8 : nt!KiPageFault+0x260
    fffff880`09995438 fffff880`0471ef35 : 00000000`0000001b fffff880`099957a0 fffffa80`55faace8 00000000`00000001 : vmswitch!memcpy+0x1e0
    fffff880`09995440 fffff880`0471f73d : 00000000`0001ff00 fffff880`09000302 00000000`0000001b fffffa80`329a3000 : vmswitch!VmsCdpCopyPortInfo+0x151
    fffff880`09995470 fffff880`046e15ee : fffff880`099957a0 00000000`00000000 fffffa80`331e0ca0 fffffa80`41b49ca0 : vmswitch!VmsCdpEnumPorts+0x1c9
    fffff880`09995640 fffff880`00f6c068 : fffffa80`331e0ca0 fffffa80`331e0df0 fffff880`099957a0 00000000`00000001 : vmswitch!VmsCdpDeviceControl+0x14e
    fffff880`09995670 fffff880`00f6b9f8 : fffffa80`331e0ca0 00000000`00000000 fffff880`099957a0 00000000`00000000 : NDIS!ndisDummyIrpHandler+0x50
    fffff880`099956a0 fffff800`a604b887 : fffffa80`41b49ca0 fffff880`09995b80 fffffa80`41b49ca0 fffff880`02e5e180 : NDIS!ndisDeviceControlIrpHandler+0xd0
    fffff880`09995890 fffff800`a6060a76 : fffffa80`370c6770 fffff880`09995a38 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x7e5
    fffff880`09995a20 fffff800`a5c75453 : fffffa80`370c6770 fffffa80`370c6770 00000000`00000000 00000000`00000000 : nt!NtDeviceIoControlFile+0x56
    fffff880`09995a90 000007fb`c4982c5a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
    000000bc`9fb1f158 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x000007fb`c4982c5a


    STACK_COMMAND:  kb

    FOLLOWUP_IP:
    vmswitch!memcpy+1e0
    fffff880`0472a7a0 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    SYMBOL_STACK_INDEX:  3

    SYMBOL_NAME:  vmswitch!memcpy+1e0

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: vmswitch

    IMAGE_NAME:  vmswitch.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  510cbec2

    BUCKET_ID_FUNC_OFFSET:  1e0

    FAILURE_BUCKET_ID:  AV_vmswitch!memcpy

    BUCKET_ID:  AV_vmswitch!memcpy

    Followup: MachineOwner
    ---------

    2: kd> lmvm vmswitch
    start             end                 module name
    fffff880`046d7000 fffff880`04768000   vmswitch   (private pdb symbols)  c:\symbolfiles\vmswitch.pdb\C3BB001449F5486880C49B8C87592D831\vmswitch.pdb
        Loaded symbol image file: vmswitch.sys
        Image path: \SystemRoot\system32\DRIVERS\vmswitch.sys
        Image name: vmswitch.sys
        Timestamp:        Fri Feb 01 23:22:42 2013 (510CBEC2)
        CheckSum:         0008C7E0
        ImageSize:        00091000
        Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4


    BarrySDCA

    Saturday, October 05, 2013 7:02 PM
  • this problem continues.  we have multiple boxes with this issue.  (all 2012 actually).  I opened Microsoft case #113100510840622

    they eventually had us re-install KB2885541, which definitely helped.  they are less frequent, but still persist.

    I updated Microsoft, and now they are saying that since the rebooting boxes are not the same exact box I mentioned in the opening ticket, they will not support it.  this is despite the fact that all boxes have the same issue, are exactly the same in regards to hardware and software, and have the same issue and bug check.  they also agreed to look at it as a whole initially but now are refusing.  the reboots are so random that waiting for that same box to reboot again is just not reasonable.

    Additionally, the support techs we worked with just seem to be sorting thru the same KB's that I did initially.  (KB2885541 was released after I opened the support ticket).

    time to start looking at other virtualization solutions.


    BarrySDCA

    Saturday, November 09, 2013 4:31 PM
  • lol these people must be kidding me.  pleased?  quality support?  nothing can be farther from the truth

    <snip>

    This is regarding a case with reference to the issue “Windows Server 2012 was rebooting unexpectedly with bugcheck 0xD1.”, on which you dealt with Gautam. While reviewing the case, I noticed that after applying hotfix for vmswitch.sys server is stable and now you have same problem on another machine and my engineer has suggested you to open a new case for that.

    Based on the above facts I assume that the case is ready for closure.  

    I want to make certain that the following two things were achieved prior to closure of this case:

    1) That you are completely pleased with the work done on this issue from Microsoft side, and
    2) That we provided you with the quality of support you deserve as a Microsoft Customer

     

    If either of the above were not achieved, I would like to hear and understand your concerns in order to take corrective measures.

    Thank you for choosing Microsoft, have a great day!


    BarrySDCA


    • Edited by BarrySDCA Wednesday, November 13, 2013 5:29 PM added CR to make it easier to read
    Wednesday, November 13, 2013 5:28 PM
  • after reloading a new OS from scratch (2012 R2), this problem STILL persists.  Intel NIC drivers have been updated too, couple times already.  MS is unable to solve it.  I am out of patience - moving away from Microsoft hypervisors. 

    Microsoft (R) Windows Debugger Version 6.2.9200.16384 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\WORKING\crash_dump\HV18\MEMORY.DMP]
    Kernel Bitmap Dump File: Only kernel address space is available

    Symbol search path is: SRV*C:\symbolfiles*http://msdl.microsoft.com/download/symbols;C:\Symbols
    Executable search path is:
    Windows 8 Kernel Version 9600 MP (16 procs) Free x64
    Product: Server, suite: TerminalServer DataCenter SingleUserTS
    Built by: 9600.16452.amd64fre.winblue_gdr.131030-1505
    Machine Name:
    Kernel base = 0xfffff801`f6e76000 PsLoadedModuleList = 0xfffff801`f713a990
    Debug session time: Mon May  5 00:53:32.355 2014 (UTC - 7:00)
    System Uptime: 18 days 16:08:42.840
    Loading Kernel Symbols
    ...............................................................
    ............................................Page 106a36 not present in the dump file. Type ".hh dbgerr004" for details
    ....................
    .............
    Loading User Symbols
    PEB is paged out (Peb.Ldr = 00007ff6`1b41c018).  Type ".hh dbgerr001" for details
    Loading unloaded module list
    ........
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {3, 2, 0, fffff800018fa260}

    Page 10e67 not present in the dump file. Type ".hh dbgerr004" for details
    Page fd2a74 not present in the dump file. Type ".hh dbgerr004" for details
    Page fd25bc not present in the dump file. Type ".hh dbgerr004" for details
    Probably caused by : vmswitch.sys ( vmswitch!memcpy+1e0 )

    Followup: MachineOwner
    ---------

    6: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 0000000000000003, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
    Arg4: fffff800018fa260, address which referenced memory

    Debugging Details:
    ------------------

    Page 10e67 not present in the dump file. Type ".hh dbgerr004" for details
    Page fd2a74 not present in the dump file. Type ".hh dbgerr004" for details
    Page fd25bc not present in the dump file. Type ".hh dbgerr004" for details

    READ_ADDRESS: fffff801f7128340: Unable to get special pool info
    fffff801f7128340: Unable to get special pool info
     0000000000000003

    CURRENT_IRQL:  2

    FAULTING_IP:
    vmswitch!memcpy+1e0
    fffff800`018fa260 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

    BUGCHECK_STR:  AV

    PROCESS_NAME:  vmms.exe

    TRAP_FRAME:  ffffd0002f0682a0 -- (.trap 0xffffd0002f0682a0)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=ffffe00013c5e2a8
    rdx=00001fffec3a1d5c rsi=0000000000000000 rdi=0000000000000000
    rip=fffff800018fa260 rsp=ffffd0002f068438 rbp=ffffd0002f068780
     r8=0000000000000004  r9=0000000000000000 r10=0000000000000000
    r11=ffffe00013c5e2a4 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    vmswitch!memcpy+0x1e0:
    fffff800`018fa260 8a440aff        mov     al,byte ptr [rdx+rcx-1] ds:00000000`00000003=??
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff801f6fcf7e9 to fffff801f6fc3ca0

    STACK_TEXT: 
    ffffd000`2f068158 fffff801`f6fcf7e9 : 00000000`0000000a 00000000`00000003 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
    ffffd000`2f068160 fffff801`f6fce03a : 00000000`00000000 00000000`00000022 00000000`00000000 ffffd000`2f0682a0 : nt!KiBugCheckDispatch+0x69
    ffffd000`2f0682a0 fffff800`018fa260 : fffff800`018eec88 00000000`00000022 ffffd000`2f068780 ffffe000`13c5d170 : nt!KiPageFault+0x23a
    ffffd000`2f068438 fffff800`018eec88 : 00000000`00000022 ffffd000`2f068780 ffffe000`13c5d170 ffffd000`20206f49 : vmswitch!memcpy+0x1e0
    ffffd000`2f068440 fffff800`01941391 : 00000000`0001ff00 ffffd000`2f000302 00000000`00000022 ffffe000`01940000 : vmswitch!VmsCdpCopyPortInfo+0x154
    ffffd000`2f068470 fffff800`0190089c : 00000000`00000000 00000000`00000000 ffffe000`17e3c510 00000000`00000001 : vmswitch!VmsCdpEnumPorts+0x1cd
    ffffd000`2f068650 fffff800`0053ff7d : ffffe000`01960430 ffffd000`2f068780 ffffe000`17e3c510 00000000`00000000 : vmswitch!VmsCdpDeviceControl+0x12be0
    ffffd000`2f068680 fffff801`f72213e5 : ffffe000`17e3c510 ffffd000`2f068b80 ffffe000`17e3c510 ffffd000`00000001 : NDIS!ndisDeviceControlIrpHandler+0x10d
    ffffd000`2f068870 fffff801`f7221d7a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x845
    ffffd000`2f068a20 fffff801`f6fcf4b3 : 00000000`00000028 00000076`f64e4a60 ffffe000`00000010 00000076`f5faf728 : nt!NtDeviceIoControlFile+0x56
    ffffd000`2f068a90 00007ffd`644565ea : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
    00000076`f5faf258 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ffd`644565ea


    STACK_COMMAND:  kb

    FOLLOWUP_IP:
    vmswitch!memcpy+1e0
    fffff800`018fa260 8a440aff        mov     al,byte ptr [rdx+rcx-1]

    SYMBOL_STACK_INDEX:  3

    SYMBOL_NAME:  vmswitch!memcpy+1e0

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: vmswitch

    IMAGE_NAME:  vmswitch.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  5253b62c

    BUCKET_ID_FUNC_OFFSET:  1e0

    FAILURE_BUCKET_ID:  AV_vmswitch!memcpy

    BUCKET_ID:  AV_vmswitch!memcpy

    Followup: MachineOwner
    ---------


    BarrySDCA

    Monday, May 05, 2014 3:23 PM
  • ps:  seems it occurs at random when a VM is stopped

    BarrySDCA

    Monday, May 05, 2014 3:40 PM
  • so in November I opened a ticket for this issue and Microsoft help was useless - absolutely embarrassing.    Well, it seems they finally published a patch:

    http://support.microsoft.com/kb/2913659/en-us

    Dated January 2014.  their patch is for only 2012 R2, which we are running now (after moving away from 2012 due to the same issue).

    If they had taken our ticket a little more seriously I'm sure they would have published the patch much sooner.

    I'm just glad we have one  - finally.

    thank you to everyone here who tried to assist.

    I should add that our boxes are up to date, but this vmswitch.sys driver does not seem to be included in the normal MS update.


    BarrySDCA


    • Marked as answer by BarrySDCA Tuesday, May 06, 2014 11:10 PM
    • Edited by BarrySDCA Tuesday, May 06, 2014 11:11 PM update vmswitch.sys info
    Tuesday, May 06, 2014 11:10 PM