none
Sysmon v11.11 Application Crashes RRS feed

  • Question

  • Hi,

    We are running Sysmon v11.11 on many windows 10 and windows server 2012/2016 and have noticed a lot of application crashes reported on a daily bases.

    Most of which seem to be related to C:\Windows\SYSTEM32\ntdll.dll but there also others like C:\Windows\System32\crypt32.dll also been seen.

    We had a quick look at the memory crash dumps and they appear to be memory related since they reference ‘heap corrupted’ or ‘memory’ in the Error codes but are not 100% sure.

    These seem to happen several times on a daily bases.

    I am hoping that someone may be able to point me into the right direction as to what to look at next or what the root cause could be.

    We are also testing version 12 in the lab to see if that resolves the issue.

    Application crash examples:

    Faulting application name: Sysmon64.exe, version: 11.11.0.0, time stamp: 0x5f0db933
    Faulting module name: ntdll.dll, version: 10.0.19041.488, time stamp: 0x70e69bad
    Exception code: 0xc0000374
    Fault offset: 0x00000000000fed79
    Faulting process ID: 0x1db4
    Faulting application start time: 0x01d6a2cf24574bab
    Faulting application path: C:\Windows\Sysmon64.exe
    Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
    Report ID: 2a117bdf-87ef-41c3-98c1-f501ae924bb5
    Faulting package full name: 
    Faulting package-relative application ID: 

    Exception Analysis                                   
    
    
    KEY_VALUES_STRING: 1
    
        Key  : Analysis.CPU.mSec
        Value: 874
    
        Key  : Analysis.DebugAnalysisProvider.CPP
        Value: Create: 8007007e on GERNOTBLT
    
        Key  : Analysis.DebugData
        Value: CreateObject
    
        Key  : Analysis.DebugModel
        Value: CreateObject
    
        Key  : Analysis.Elapsed.mSec
        Value: 1214
    
        Key  : Analysis.Memory.CommitPeak.Mb
        Value: 115
    
        Key  : Analysis.System
        Value: CreateObject
    
        Key  : Timeline.Process.Start.DeltaSec
        Value: 2122
    
        Key  : WER.OS.Branch
        Value: vb_release
    
        Key  : WER.OS.Timestamp
        Value: 2019-12-06T14:06:00Z
    
        Key  : WER.OS.Version
        Value: 10.0.19041.1
    
        Key  : WER.Process.Version
        Value: 14.3.227.0
    
    
    ADDITIONAL_XML: 1
    
    OS_BUILD_LAYERS: 1
    
    NTGLOBALFLAG:  0
    
    PROCESS_BAM_CURRENT_THROTTLED: 0
    
    PROCESS_BAM_PREVIOUS_THROTTLED: 0
    
    APPLICATION_VERIFIER_FLAGS:  0
    
    CONTEXT:  (.ecxr)
    rax=00007ffa3be695f1 rbx=00000000c0000374 rcx=000000c9d38fe8b0
    rdx=00007ffa3be695f1 rsi=0000000000000001 rdi=00007ffa3bfb77f0
    rip=00007ffa3bf4ed79 rsp=000000c9d38fe5c0 rbp=0000000000800000
     r8=0000000000000000  r9=0000026eee63a650 r10=00007ffa0000001d
    r11=000000c9d38fe3a0 r12=0000000000000001 r13=0000000000000000
    r14=0000026eee259d10 r15=0000026eee259b00
    iopl=0         nv up ei pl nz na po nc
    cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
    ntdll!RtlReportFatalFailure+0x9:
    00007ffa`3bf4ed79 eb00            jmp     ntdll!RtlReportFatalFailure+0xb (00007ffa`3bf4ed7b)
    Resetting default scope
    
    EXCEPTION_RECORD:  (.exr -1)
    ExceptionAddress: 00007ffa3bf4ed79 (ntdll!RtlReportFatalFailure+0x0000000000000009)
       ExceptionCode: c0000374
      ExceptionFlags: 00000001
    NumberParameters: 1
       Parameter[0]: 00007ffa3bfb77f0
    
    PROCESS_NAME:  Sysmon64.exe
    
    ERROR_CODE: (NTSTATUS) 0xc0000374 - A heap has been corrupted.
    
    EXCEPTION_CODE_STR:  c0000374
    
    EXCEPTION_PARAMETER1:  00007ffa3bfb77f0
    
    ADDITIONAL_DEBUG_TEXT:  Enable Pageheap/AutoVerifer ; Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[PSEUDO_THREAD]
    
    FAULTING_THREAD:  00005a5c
    
    STACK_TEXT:  
    00000000`00000000 00000000`00000000 heap_corruption!Sysmon64.exe+0x0
    
    
    SYMBOL_NAME:  heap_corruption!Sysmon64.exe
    
    MODULE_NAME: heap_corruption
    
    IMAGE_NAME:  heap_corruption
    
    STACK_COMMAND:  ** Pseudo Context ** ManagedPseudo ** Value: 275803fc3f0 ** ; kb
    
    FAILURE_BUCKET_ID:  HEAP_CORRUPTION_c0000374_heap_corruption!Sysmon64.exe
    
    OS_VERSION:  10.0.19041.1
    
    BUILDLAB_STR:  vb_release
    
    OSPLATFORM_TYPE:  x64
    
    OSNAME:  Windows 10
    
    FAILURE_ID_HASH:  {50d0a67e-f672-6263-4f04-64af1dcce6ef}

    Faulting application name: Sysmon64.exe, version: 11.11.0.0, time stamp: 0x5f0db933
    Faulting module name: CRYPT32.dll, version: 10.0.19041.21, time stamp: 0xeed852a5
    Exception code: 0xc0000005
    Fault offset: 0x0000000000041cc9
    Faulting process ID: 0x4540
    Faulting application start time: 0x01d6a20866cf7bfc
    Faulting application path: C:\Windows\Sysmon64.exe
    Faulting module path: C:\Windows\System32\CRYPT32.dll
    Report ID: 146ca02e-b605-4976-b077-7b3fc562cc7a
    Faulting package full name: 
    Faulting package-relative application ID: 

    Exception Analysis                                   
    
    KEY_VALUES_STRING: 1
    
        Key  : AV.Dereference
        Value: NullPtr
    
        Key  : AV.Fault
        Value: Read
    
        Key  : Analysis.CPU.mSec
        Value: 577
    
        Key  : Analysis.DebugAnalysisProvider.CPP
        Value: Create: 8007007e on GERNOTBLT
    
        Key  : Analysis.DebugData
        Value: CreateObject
    
        Key  : Analysis.DebugModel
        Value: CreateObject
    
        Key  : Analysis.Elapsed.mSec
        Value: 956
    
        Key  : Analysis.Memory.CommitPeak.Mb
        Value: 117
    
        Key  : Analysis.System
        Value: CreateObject
    
        Key  : Timeline.Process.Start.DeltaSec
        Value: 80366
    
        Key  : WER.OS.Branch
        Value: vb_release
    
        Key  : WER.OS.Timestamp
        Value: 2019-12-06T14:06:00Z
    
        Key  : WER.OS.Version
        Value: 10.0.19041.1
    
        Key  : WER.Process.Version
        Value: 14.3.227.0
    
    
    ADDITIONAL_XML: 1
    
    OS_BUILD_LAYERS: 1
    
    NTGLOBALFLAG:  0
    
    PROCESS_BAM_CURRENT_THROTTLED: 0
    
    PROCESS_BAM_PREVIOUS_THROTTLED: 0
    
    APPLICATION_VERIFIER_FLAGS:  0
    
    CONTEXT:  (.ecxr)
    rax=0000022799ab3df0 rbx=0000000000000000 rcx=0000000000000000
    rdx=0000000000000000 rsi=000002279978a570 rdi=000002279978a570
    rip=00007ffa39971cc9 rsp=000000c69f5fef08 rbp=0000000000000002
     r8=0000000000000000  r9=0000000000000002 r10=0000022799459cd2
    r11=000002279978a580 r12=000000c69f5fef80 r13=0000000000000000
    r14=0000022799ab3de0 r15=000002279978a580
    iopl=0         nv up ei pl nz na po nc
    cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010206
    crypt32!ChainIsOIDInUsage+0x21:
    00007ffa`39971cc9 0fb602          movzx   eax,byte ptr [rdx] ds:00000000`00000000=??
    Resetting default scope
    
    EXCEPTION_RECORD:  (.exr -1)
    ExceptionAddress: 00007ffa39971cc9 (crypt32!ChainIsOIDInUsage+0x0000000000000021)
       ExceptionCode: c0000005 (Access violation)
      ExceptionFlags: 00000000
    NumberParameters: 2
       Parameter[0]: 0000000000000000
       Parameter[1]: 0000000000000000
    Attempt to read from address 0000000000000000
    
    PROCESS_NAME:  Sysmon64.exe
    
    READ_ADDRESS:  0000000000000000 
    
    ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.
    
    EXCEPTION_CODE_STR:  c0000005
    
    EXCEPTION_PARAMETER1:  0000000000000000
    
    EXCEPTION_PARAMETER2:  0000000000000000
    
    STACK_TEXT:  
    000000c6`9f5fef08 00007ffa`39971c63     : 00000000`00000000 00007ffa`39970000 00000000`00000000 01d6a2c3`84209222 : crypt32!ChainIsOIDInUsage+0x21
    000000c6`9f5fef10 00007ffa`3997298d     : 00000000`00000000 000000c6`9f5fefa0 00000227`99ab3de0 00000000`00000000 : crypt32!ChainAppendUsages+0x8b
    000000c6`9f5fef60 00007ffa`3997269b     : 000000c6`00000000 000000c6`00000001 00000227`00000000 000000c6`9f5ff0e0 : crypt32!ChainCalculateRestrictedUsage+0x159
    000000c6`9f5fefe0 00007ffa`39971eb1     : 00000000`00000000 00000227`9818a580 00000227`989468a8 00000227`989468e8 : crypt32!CChainPathObject::UpdateChainContextUsageForPathObject+0x1c7
    000000c6`9f5ff0a0 00007ffa`39974ce3     : ecd52fc5`00000001 00000227`9818a580 00000000`00000001 00000227`9954bca0 : crypt32!CChainPathObject::CreateChainContextFromPath+0x1b1
    000000c6`9f5ff180 00007ffa`3995bce1     : 00000227`00000000 000000c6`00000001 00000227`993b0da0 00000227`9960b210 : crypt32!CCertChainEngine::CreateChainContextFromPathGraph+0x2bb
    000000c6`9f5ff2f0 00007ffa`3995bed8     : 00000227`9936fa80 00000227`993b0da0 00000000`00000000 00000227`9960b210 : crypt32!CCertChainEngine::GetChainContext+0x91
    000000c6`9f5ff380 00007ffa`397ba938     : 00000227`9936fa80 00000000`00000000 00000227`99814214 00007ffa`397f5ab0 : crypt32!CertGetCertificateChain+0xf8
    000000c6`9f5ff420 00007ffa`397ba6c2     : 000000c6`9f5ff640 00000000`00000000 000000c6`9f5ff5a0 00000000`00000000 : wintrust!_WalkChain+0x1f8
    000000c6`9f5ff530 00007ffa`397b268e     : 00000000`00000000 00000000`00000000 000000c6`9f5ff6a0 00000000`00000000 : wintrust!WintrustCertificateTrust+0xb2
    000000c6`9f5ff5a0 00007ffa`397b1de5     : 00000000`00000002 00007ffa`3be695f1 00000227`997948a0 00000000`0000001b : wintrust!I_VerifyTrust+0x86e
    000000c6`9f5ff910 00007ff7`4cdb5344     : 00000000`00000000 00000000`00000040 00000000`00000017 00000227`99cee020 : wintrust!WinVerifyTrust+0x45
    000000c6`9f5ff950 00000000`00000000     : 00000000`00000040 00000000`00000017 00000227`99cee020 00000000`00000017 : Sysmon64+0x35344
    
    
    SYMBOL_NAME:  crypt32!ChainIsOIDInUsage+21
    
    MODULE_NAME: crypt32
    
    IMAGE_NAME:  crypt32.dll
    
    STACK_COMMAND:  ~4s ; .ecxr ; kb
    
    FAILURE_BUCKET_ID:  NULL_POINTER_READ_c0000005_crypt32.dll!ChainIsOIDInUsage
    
    OS_VERSION:  10.0.19041.1
    
    BUILDLAB_STR:  vb_release
    
    OSPLATFORM_TYPE:  x64
    
    OSNAME:  Windows 10
    
    IMAGE_VERSION:  10.0.19041.21
    
    FAILURE_ID_HASH:  {46de1d98-a625-ff2d-634a-15de71d43d99}

    Many thanks in advance,

    Gernot Baar


    • Edited by Gernot_Baar Thursday, October 15, 2020 12:31 PM
    Thursday, October 15, 2020 12:29 PM

All replies