Server 2012 Hyper-V Host Crashes with BAD_POOL_CALLER need some help with fine tune analysis

Answered Server 2012 Hyper-V Host Crashes with BAD_POOL_CALLER need some help with fine tune analysis

  • Wednesday, December 05, 2012 6:38 PM
     
     

    Hello everyone,

    I'm new to analyzing crash dumps and so far have come up with that it is likely I had some sort of memory problem or VHD issue with vhdmp.sys.  When it crashed there were many servers re synchronizing with their replicas which is hosted on a SMB 3.0 share on a server 2012 standard edition file server.  I know there likely was a lot of disk I/O and there are 12 VMs on it, 3 of them are rather big ones server about 2,000 people for video streaming and delivery.

    System Info:

    Mobo: SuperMicro X9DRI-LN4F+ Intel Xeon DualProc

    Proc: 2x Intel Xeon-SandyBridge E5-2690-OctoCore

    Memory: 8x Samsung 1600 16GB DDR3 2Rx4 [16GB]

    Raid Controller: Adaptec 5405 Z (1b4611d360f)

    Battery : Adaptec ZMM-100CC

    OS SSD: Intel x25-E Extreme ( SSDSA2SH064G1)

    Raid 50 Array for Hyper-V:

    10X Seagate ConstellationES.2 ST33000650NS [3000GB]

    10 gbps fiber nic:

    SuperMicro AOC-STGN-i2S SuperMicro 10 Gigabit Port

    Power Supply: 2x 
    SuperMicro PWS-920P-1R 920W (one is a redundant power supply )

    Here is my !analyze -v output from the debugger:

    ---------------------------------------------------------------------------

    Microsoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\Users\MatthewMobile\Desktop\Kernel_0_0_cab_108c3987\120412-14414-01.dmp]
    Mini Kernel Dump File: Only registers and stack trace are available

    Symbol search path is: SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols
    Executable search path is: 
    Windows 8 Kernel Version 9200 MP (32 procs) Free x64
    Product: Server, suite: TerminalServer DataCenter SingleUserTS
    Built by: 9200.16424.amd64fre.win8_gdr.120926-1855
    Machine Name:
    Kernel base = 0xfffff801`a528b000 PsLoadedModuleList = 0xfffff801`a5555a60
    Debug session time: Tue Dec  4 18:42:51.756 2012 (UTC - 8:00)
    System Uptime: 19 days 8:49:44.110
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ..........................
    Loading User Symbols
    Loading unloaded module list
    .................
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck C2, {7, 11c1, 4200016, fffffa80a33cf750}

    GetPointerFromAddress: unable to read from fffff801a55e1168
    GetUlongFromAddress: unable to read from fffff801a55e11f8
    Probably caused by : vhdmp.sys ( vhdmp!VhdmpiFreeVirtualDiskSurface+6f )

    Followup: MachineOwner
    ---------

    12: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    BAD_POOL_CALLER (c2)
    The current thread is making a bad pool request.  Typically this is at a bad IRQL level or double freeing the same allocation, etc.
    Arguments:
    Arg1: 0000000000000007, Attempt to free pool which was already freed
    Arg2: 00000000000011c1, (reserved)
    Arg3: 0000000004200016, Memory contents of the pool block
    Arg4: fffffa80a33cf750, Address of the block of pool being deallocated

    Debugging Details:
    ------------------


    POOL_ADDRESS:  fffffa80a33cf750 Nonpaged pool

    FREED_POOL_TAG:  VHDu

    BUGCHECK_STR:  0xc2_7_VHDu

    CUSTOMER_CRASH_COUNT:  1

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT_SERVER

    PROCESS_NAME:  System

    CURRENT_IRQL:  0

    LAST_CONTROL_TRANSFER:  from fffff801a54fba5a to fffff801a5305d40

    STACK_TEXT:  
    fffff880`061469c8 fffff801`a54fba5a : 00000000`000000c2 00000000`00000007 00000000`000011c1 00000000`04200016 : nt!KeBugCheckEx
    fffff880`061469d0 fffff880`09c4a563 : fffffa80`a33cf750 00000000`00000000 fffffa80`a15b5040 fffff880`09c7c110 : nt!ExDeferredFreePool+0xe86
    fffff880`06146ab0 fffff880`09c4a7c5 : fffffa80`a33cf750 00000000`00000000 00000000`00000000 fffff801`a530f4c4 : vhdmp!VhdmpiFreeVirtualDiskSurface+0x6f
    fffff880`06146ae0 fffff880`09c4ad8b : fffffa80`a33cf750 fffff880`09c7c110 fffffa80`a33cf918 fffff801`a530f4c4 : vhdmp!VhdmpiDecoupleVirtualDiskSurface+0x16d
    fffff880`06146b10 fffff880`09c3ad49 : fffffa80`a33cf8f8 fffff880`09c889a0 fffff801`a550f110 fffffffe`9a5f4400 : vhdmp!VhdmpiHaltActiveSurface+0x123
    fffff880`06146b50 fffff801`a533e591 : fffff801`a550f110 fffffa80`977c9b00 fffff801`a52f1300 fffff801`a530f400 : vhdmp!VhdmpiSynchronousWorkerRoutineWrapper+0x19
    fffff880`06146b80 fffff801`a52ad535 : fffffa80`64a04660 00000000`00000080 fffff801`a533e450 fffffa80`977c9b00 : nt!ExpWorkerThread+0x142
    fffff880`06146c10 fffff801`a52ebe16 : fffff880`03177180 fffffa80`977c9b00 fffff880`03183540 fffffa80`61d0f340 : nt!PspSystemThreadStartup+0x59
    fffff880`06146c60 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16


    STACK_COMMAND:  kb

    FOLLOWUP_IP: 
    vhdmp!VhdmpiFreeVirtualDiskSurface+6f
    fffff880`09c4a563 488b0da61b0300  mov     rcx,qword ptr [vhdmp!WPP_GLOBAL_Control (fffff880`09c7c110)]

    SYMBOL_STACK_INDEX:  2

    SYMBOL_NAME:  vhdmp!VhdmpiFreeVirtualDiskSurface+6f

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: vhdmp

    IMAGE_NAME:  vhdmp.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  5010aaa8

    BUCKET_ID_FUNC_OFFSET:  6f

    FAILURE_BUCKET_ID:  0xc2_7_VHDu_vhdmp!VhdmpiFreeVirtualDiskSurface

    BUCKET_ID:  0xc2_7_VHDu_vhdmp!VhdmpiFreeVirtualDiskSurface

    Followup: MachineOwner
    ---------

    ---------------------------------------------------------------------------

    Thank you all so much for any help, advice or direction!!

All Replies

  • Thursday, December 06, 2012 6:18 AM
    Moderator
     
     Answered

    Hi,

    This Forum is not the best place for dump file analyzing. It’s suggested to contact Microsoft Customer Support Services (CSS) so that a dedicated Support Professional can help you on this issue.

    To obtain the phone numbers for specific technology request, please refer to the website listed below:

    http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS

    If you are outside the US, please refer to http://support.microsoft.com  for regional support phone numbers.

    Thanks for your understanding.


    Lawrence

    TechNet Community Support

  • Monday, February 11, 2013 3:57 PM
     
     
    Did you ever figure out what was causing your crashes?  I am running Server 2012 (standard) with Hyper-V installed, but I am not currently using Hyper-V.  I was using it (Hyper-V), but thought it was causing these crashes, so I stopped all VMs and moved to another Server.  I continue to get these crashes after stopping all VMs.  I have had 4 spontaneous crashes in the last month.