none
Windows2012R2 에서 가상OS(hyper-v) 가 계속 랜덤으로 재부팅 됩니다.(windbg 분석한자료 첨부합니다.) RRS feed

  • 질문

  • 1. 환경

      가) 장비 : DL380G9

      나) OS : Windows2012R2 Dacenter Edition

                : hyper-V 게스트 OS => Windows2012R2 Standard 약 4대(2세대), CentOS7 ( 1세대)

      다) 증상 : 4대의 Hyper-V 는 정상적인데, 한개의 게스트OS 만 랜덤으로 재부팅됨.

    가상화OS 의 이벤트 로그 및 내용은 아래와 같습니다.

    event id 1001 (BugCheck)컴퓨터가 오류 검사 후 다시 부팅되었습니다. 
    오류 검사: 0x000000d1 (0x000000000000003c, 0x0000000000000002, 0x0000000000000001, 0xfffff80004298862).

    덤프 저장 위치: c:Windows-MEMORY.DMP 보고서 ID 122917-16203-01


    event id 41 (kernel power)
    시스템이 비정상적으로 종료된 후 다시 부팅되었습니다. 이 오류는 시스템이 응답을 멈추었거나
    손상되었거나 예기치 않게 전원 공급이 중단되면 발생할 수 있습니다.


    Windbg 를 이용한 메모리 덤프 내용은 아래와 같습니다.


    Microsoft (R) Windows Debugger Version 10.0.15063.468 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\UTIL\IMSI\MEMORY.DMP]
    Kernel Bitmap Dump File: Kernel address space is available, User address space may not be available.

    Symbol search path is: srv*
    Executable search path is:
    Windows 8.1 Kernel Version 9600 MP (4 procs) Free x64
    Product: Server, suite: TerminalServer SingleUserTS
    Built by: 9600.18821.amd64fre.winblue_ltsb.170914-0600
    Machine Name:
    Kernel base = 0xfffff802`e2005000 PsLoadedModuleList = 0xfffff802`e22d7650
    Debug session time: Fri Jan  5 23:59:49.086 2018 (UTC + 9:00)
    System Uptime: 0 days 7:17:56.087
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ....
    Loading User Symbols

    Loading unloaded module list
    ...........
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {3c, 2, 1, fffff801ba92c862}

    *** ERROR: Module load completed but symbols could not be loaded for netvsc63.sys
    *** ERROR: Symbol file could not be found.  Defaulted to export symbols for vmbkmcl.sys -
    *** ERROR: Module load completed but symbols could not be loaded for vmbus.sys
    Probably caused by : netvsc63.sys ( netvsc63+2aa1 )

    Followup:     MachineOwner
    ---------

    0: kd>
    0: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 000000000000003c, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000001, value 0 = read operation, 1 = write operation
    Arg4: fffff801ba92c862, address which referenced memory

    Debugging Details:
    ------------------


    DUMP_CLASS: 1

    DUMP_QUALIFIER: 401

    BUILD_VERSION_STRING:  9600.18821.amd64fre.winblue_ltsb.170914-0600

    SYSTEM_MANUFACTURER:  Microsoft Corporation

    VIRTUAL_MACHINE:  HyperV

    SYSTEM_PRODUCT_NAME:  Virtual Machine

    SYSTEM_SKU:  None

    SYSTEM_VERSION:  Hyper-V UEFI Release v1.0

    BIOS_VENDOR:  Microsoft Corporation

    BIOS_VERSION:  Hyper-V UEFI Release v1.0

    BIOS_DATE:  11/26/2012

    BASEBOARD_MANUFACTURER:  Microsoft Corporation

    BASEBOARD_PRODUCT:  Virtual Machine

    BASEBOARD_VERSION:  Hyper-V UEFI Release v1.0

    DUMP_TYPE:  1

    BUGCHECK_P1: 3c

    BUGCHECK_P2: 2

    BUGCHECK_P3: 1

    BUGCHECK_P4: fffff801ba92c862

    WRITE_ADDRESS:  000000000000003c

    CURRENT_IRQL:  2

    FAULTING_IP:
    tcpip!TcpBeginTcbSend+732
    fffff801`ba92c862 f0ff403c        lock inc dword ptr [rax+3Ch]

    CPU_COUNT: 4

    CPU_MHZ: bb5

    CPU_VENDOR:  GenuineIntel

    CPU_FAMILY: 6

    CPU_MODEL: 4f

    CPU_STEPPING: 1

    CPU_MICROCODE: 6,4f,1,0 (F,M,S,R)  SIG: FFFFFFFF'00000000 (cache) FFFFFFFF'00000000 (init)

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

    BUGCHECK_STR:  AV

    PROCESS_NAME:  System

    ANALYSIS_SESSION_HOST:  SKY_NICE

    ANALYSIS_SESSION_TIME:  01-08-2018 22:08:27.0306

    ANALYSIS_VERSION: 10.0.15063.468 amd64fre

    TRAP_FRAME:  fffff802e38d55f0 -- (.trap 0xfffff802e38d55f0)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=ffffe000f86fb780
    rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff801ba92c862 rsp=fffff802e38d5780 rbp=fffff802e38d5880
     r8=ffffe000f86fb770  r9=ffffe000f86fb6b0 r10=ffffe000f86fb6b0
    r11=ffffe000f86fb8c8 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    tcpip!TcpBeginTcbSend+0x732:
    fffff801`ba92c862 f0ff403c        lock inc dword ptr [rax+3Ch] ds:00000000`0000003c=????????
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff802e215e6e9 to fffff802e2152ba0

    STACK_TEXT: 
    fffff802`e38d54a8 fffff802`e215e6e9 : 00000000`0000000a 00000000`0000003c 00000000`00000002 00000000`00000001 : nt!KeBugCheckEx
    fffff802`e38d54b0 fffff802`e215cf3a : 00000000`00000001 fffff802`e38d5838 ffffe000`f5f2bc00 fffff801`bc0b5d5a : nt!KiBugCheckDispatch+0x69
    fffff802`e38d55f0 fffff801`ba92c862 : 00000000`fffffffe ffffe000`f8781360 ffffe000`f8781360 00000000`000001ff : nt!KiPageFault+0x23a
    fffff802`e38d5780 fffff801`ba92bc2f : ffffe000`f6b1c4c0 00000000`00000000 00000000`5f636357 ffffe000`f8781360 : tcpip!TcpBeginTcbSend+0x732
    fffff802`e38d5a60 fffff801`ba9475c1 : 00000000`00000002 00000000`00000001 ffffe000`f6b29470 ffffe000`f612fd80 : tcpip!TcpTcbSend+0x5df
    fffff802`e38d5d50 fffff801`ba92357c : ffffe000`00000000 00000000`0028181a 00000000`00000000 fffff802`e38d6200 : tcpip!TcpFlushDelay+0x1c1
    fffff802`e38d5df0 fffff801`ba91d0b3 : ffffe000`f61abf00 00000000`00005000 00000000`00006ffe 00000000`00006ffe : tcpip!TcpPreValidatedReceive+0x3cc
    fffff802`e38d5ef0 fffff801`ba949e72 : ffffe000`f6685260 fffff802`e38d6380 fffff801`00000006 ffffe000`f6ad0006 : tcpip!IpFlcReceivePreValidatedPackets+0x649
    fffff802`e38d60d0 fffff802`e20bab83 : fffff802`e38d6250 00000000`00000000 ffffe000`f6183c10 fffff802`e38d1000 : tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x102
    fffff802`e38d6200 fffff801`ba94a0b6 : fffff801`ba949d70 fffff802`e38d6320 0000000e`f636e110 ffffe000`f6ad9704 : nt!KeExpandKernelStackAndCalloutInternal+0xf3
    fffff802`e38d62f0 fffff801`ba0f3a53 : 00000000`00000000 fffff802`e38d63d1 00000000`00000001 fffff801`ba1032a5 : tcpip!FlReceiveNetBufferListChain+0xb6
    fffff802`e38d6370 fffff801`ba0f3e7f : ffffe000`f6b29401 fffff802`e38d0008 00000000`00000000 ffffe000`00000001 : NDIS!ndisMIndicateNetBufferListsToOpen+0x123
    fffff802`e38d6430 fffff801`ba0f46b2 : ffffe000`f636e1a0 ffffe000`f6b1c401 fffff801`ba100540 fffff802`00000000 : NDIS!ndisMTopReceiveNetBufferLists+0x22f
    fffff802`e38d64c0 fffff801`b9fcfaa1 : ffffe000`f6b1c4c0 ffff223f`3ab53783 00000000`00000001 fffff801`ba3ac101 : NDIS!NdisMIndicateReceiveNetBufferLists+0x732
    fffff802`e38d66b0 fffff801`b9fcf8fb : 00000000`00000000 ffffe000`f6b1c6e0 00000000`0000ff02 fffff801`00000000 : netvsc63+0x2aa1
    fffff802`e38d6750 fffff801`ba3bfd61 : 00000000`0000001e fffff801`ba3ac101 ffffe000`f5fc1590 00000000`00000040 : netvsc63+0x28fb
    fffff802`e38d67c0 fffff801`ba3a22c9 : ffffe000`f65c26c0 00000000`00000001 00000000`00000000 00000000`00000000 : vmbkmcl!VmbChannelPacketComplete+0xa01
    fffff802`e38d6830 fffff802`e2036e50 : fffff802`e2303f00 fffff802`e38d6990 fffff802`e38d6b20 fffff802`e279658f : vmbus+0x12c9
    fffff802`e38d6890 fffff802`e2036197 : 00000000`00000000 ffffe000`f7a81080 fffff802`e2301180 fffff802`00000000 : nt!KiExecuteAllDpcs+0x1b0
    fffff802`e38d69e0 fffff802`e21566ea : fffff802`e2301180 fffff802`e2301180 fffff802`e2359a00 ffffe000`f7a86080 : nt!KiRetireDpcList+0xd7
    fffff802`e38d6c60 00000000`00000000 : fffff802`e38d7000 fffff802`e38d1000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x5a


    STACK_COMMAND:  kb

    THREAD_SHA1_HASH_MOD_FUNC:  0d7cc75dac4954dbded7f32b52612857e2b7b09a

    THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  4a5a48b9905ea92b296339cf0b55de7466ae4bd0

    THREAD_SHA1_HASH_MOD:  0bc77d3bbc34aa415b82bf8ea7f4c89913f6cac1

    FOLLOWUP_IP:
    netvsc63+2aa1
    fffff801`b9fcfaa1 f6c302          test    bl,2

    FAULT_INSTR_CODE:  f02c3f6

    SYMBOL_STACK_INDEX:  e

    SYMBOL_NAME:  netvsc63+2aa1

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: netvsc63

    IMAGE_NAME:  netvsc63.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  590e16a4

    BUCKET_ID_FUNC_OFFSET:  2aa1

    FAILURE_BUCKET_ID:  AV_netvsc63!unknown_function

    BUCKET_ID:  AV_netvsc63!unknown_function

    PRIMARY_PROBLEM_CLASS:  AV_netvsc63!unknown_function

    TARGET_TIME:  2018-01-05T14:59:49.000Z

    OSBUILD:  9600

    OSSERVICEPACK:  0

    SERVICEPACK_NUMBER: 0

    OS_REVISION: 0

    SUITE_MASK:  272

    PRODUCT_TYPE:  3

    OSPLATFORM_TYPE:  x64

    OSNAME:  Windows 8.1

    OSEDITION:  Windows 8.1 Server TerminalServer SingleUserTS

    OS_LOCALE: 

    USER_LCID:  0

    OSBUILD_TIMESTAMP:  2017-09-14 22:34:00

    BUILDDATESTAMP_STR:  170914-0600

    BUILDLAB_STR:  winblue_ltsb

    BUILDOSVER_STR:  6.3.9600.18821.amd64fre.winblue_ltsb.170914-0600

    ANALYSIS_SESSION_ELAPSED_TIME:  a19

    ANALYSIS_SOURCE:  KM

    FAILURE_ID_HASH_STRING:  km:av_netvsc63!unknown_function

    FAILURE_ID_HASH:  {26e500c1-f3c8-d17d-50fe-5d6f84e72538}

    Followup:     MachineOwner
    ---------

    앞에 똑같은 질문에 답주신대로, 일단 해보고는 있는데.....

    아직 사이트에 적용하진 못하고 있습니다.


    • 편집됨 oceanheli 2018년 1월 8일 월요일 오후 1:12
    2018년 1월 8일 월요일 오후 1:06

답변

모든 응답

  • 안녕하세요?

    1. netvsc63.sys에 의하여 문제가 발생한 것으로 보여지며 해당 파일은 C:\Windows\system32\driver 폴더에 위치한 Windows 2012 R2에 포함된 파일로 보여집니다.

    2. 이에 Windows Update를 진행하여 주시고 백신의 엔진 업데이트 하여 주시기 바랍니다.

    3. 하기 핫픽스도 적용해 보시기 바랍니다.

    0x0000007F Stop error in nlb.sys, ndis.sys, or netvsc63.sys in Windows Server 2012 R2

    https://support.microsoft.com/en-us/help/2953561/0x0000007f-stop-error-in-nlb-sys--ndis-sys--or-netvsc63-sys-in-windows

    감사합니다.

    2018년 1월 9일 화요일 오전 1:40
  • 1. Windows Update 는 당연 진행하였고, 백신(Ahnlab)도 최신으로 업데이트하고 바이러스 검사를 하였습니다.

    2. 알려주신 핫픽스를 적용했는데(게스트OS) , 증상이 똑같네요 ㅠ.ㅠ

    더 해볼수 있는 방법은 없는지요....

    특이한건, 이 게스트OS 는 원래 다른서버에서 문제가 생겨 재설치 했습니다.

    그런데도, 유독 hyper-v 에서 이 게스트OS 만 계속 재부팅 현상이 있습니다.

    2018년 1월 10일 수요일 오전 11:51
  • 안녕하세요?

    이제는 사례기반, 일부 덤프정보 기반으로는 분석이 어려울 것으로 보여집니다.

    이에 필요하시면 한국 마이크로소프트 고객지원센터 1577-9700번으로 전화하셔서 유료로 C:\Windows\memory.dmp을 분석 받으셔야 할 것으로 보여집니다.

    감사합니다.

    2018년 1월 11일 목요일 오전 12:11