locked
Crash dump / minidump on Lightweight gateways filling up system volume RRS feed

  • Question

  • Hello, recently we noticed our system volumes filling up on a few domain controllers in our environment. The lightweight gateways are experiencing a what appears to be memory related crash/dump. Some DC's did not experience this at all others had anywhere from 4 to 6 .dmp files ranging in size from 8MB to 11GB.

    When we analyze the dump we are given the following info :



    Exception Information

    CLR!_CHKSTK+37In Microsoft.Tri.Gateway.exe.9328.dmp the assembly instruction at <g class="gr_ gr_575 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" data-gr-id="575" id="575">clr</g>!_chkstk+37 in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll from Microsoft Corporation has caused a stack overflow exception (0xC00000FD) when trying to write to memory location 0x8061a000 on thread 58
    Module Information

    Image Name: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll  Symbol Type: PDB
    Base address: 0x00000003`00905a4d  Time Stamp: Tue Nov 29 21:06:30 2016
    Checksum: 0x00000000`00000000  Comments: Flavor=Retail
    COM DLL: False  Company Name: Microsoft Corporation
    ISAPIExtension: False  File Description: Microsoft .NET Runtime Common Language Runtime - WorkStation
    ISAPIFilter: False  File Version: 4.6.1087.0 built by: NETFXREL4STAGE
    Managed DLL: False  Internal Name: clr.dll
    VB DLL: False  Legal Copyright: © Microsoft Corporation. All rights reserved.
    Loaded Image Name: clr.dll  Legal Trademarks:
    Mapped Image Name: c:\symcache\clr.dll\583E5E56990000\clr.dll  Original filename: clr.dll
    Module name: clr  Private Build: DDBLD295H
    Single Threaded: False  Product Name: Microsoft® .NET Framework
    Module Size: 9.56 MBytes  Product Version: 4.6.1087.0
    Symbol File Name: c:\symcache\clr.pdb\3517C9FE1EF04FEB810112EC139D63762\clr.pdb  Special Build: &




    Is anyone else experiencing this issue or have advice on how to further troubleshoot it?

    Can we adjust the dump/minidump settings on the systems or disable this behavior if needed?


    Tuesday, September 19, 2017 8:14 PM

All replies

  • That's odd.

    Crash Dumps are disabled by default for LWGW.

    Any chance you have another debug settings set at machine level that will turn on dumps?

    Can you dump the registry's sub tree of SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps

    And paste it here?

    Tuesday, September 19, 2017 8:28 PM
  • On one of the machines with a dump this exists :

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\Microsoft.Tri.Gateway.exe

    at this path in the registry, there is a Default string there with no value set.

    There is also a DWORD String "DumpType" with a hex value of 2.

    This also exists with the same values on a system without any dumps, leading us to believe it's set this way on all DCs yet only some are 'crashing'.





    • Edited by niiiiiick Tuesday, September 19, 2017 10:08 PM
    Tuesday, September 19, 2017 9:20 PM
  • It sounds like we can disable the minidumps by deleting the reg key below? Is there any explanation as to why this is "enabled" on our DCs with the lightweight gateway?

    We're also willing to help with debugging the dumps themselves, they appear to a couple times a week randomly on our DCs.

    Wednesday, September 20, 2017 4:20 PM
  • Any chance you installed the Gateway while the machines were not promoted?

    Yes, deleting the key should avoid the dump, but it's important to understand why the key was generated, as it might come back for the same reason in future updates.

    as for the dumps, if you email me at atashare at microsoft com, I can provide a secured workspace where

    you can upload the dumps and I can take a look, or you can open a case with Microsoft Support for ATA.

    Also, it would be great if you can zip the textual logs under the Logs subfolder and upload as well.

    Eli.

    Wednesday, September 20, 2017 6:05 PM
  • All the systems were already promoted to DC's. We installed on ATA 1.7 and upgraded to 1.8, pushing out the upgrade to the lightweight gateways in late July. Is there anything we can do to assist with the cause? We are in agreement about keeping it disabled permanently.

    I will send correspondence with the memory dumps.

    Thank you!

    Wednesday, September 20, 2017 6:18 PM
  • If the systems were already  promoted it reduces the chance this key was created by ATA.

    I had a case before where the customer had a 3rd  server monitoring app that did that.

    Can you tell me where on the disk the dump files are created?

    It's also weird, because when our code generates this key for a standalone GW scenario,

    under the executable name, we create 3 values: DumpFolder (which is set to put the files under out folder), DumpCount which limits the system to keep only 3 files, and DumpType, which indeed we set to 2.

    The fact that you are missing DumpFolder & DumpCount is partially why you are running out of space,

    but also indicates that it might noe have been ATA creating these dumps.

    The location on the disk of the dump files might give a clue about who creates them.

    Wednesday, September 20, 2017 6:32 PM
  • The dumps are always at this path :

    c:\Windows\ServiceProfiles\LocalService\AppData\Local\CrashDumps

    We only assume it's the gateway due to the filenames :

    <g class="gr_ gr_157 gr-alert gr_gramm gr_hide gr_inline_cards gr_run_anim Style multiReplace replaceWithoutSep" data-gr-id="157" id="157">Microsoft.Tri.Gateway.exe.9328.dmp ,</g> where '9328' is the PID of the process.

    Thursday, September 21, 2017 3:21 PM
  • This is not the path we are setting, but this is expected since the registry values are different from what we set.

    Are there any other keys under

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps

    besides the Gateway?


    Thursday, September 21, 2017 6:47 PM
  • If you can post these files the dumps can be debugged:

    1) mini dump files:  %SystemRoot%\Minidump or c:\windows\minidump

    2) memory dump file:  %SystemRoot%\MEMORY.DMP or C:\Windows\MEMORY.DMP

    Use file explorer > this PC > local C: drive > right upper corner search enter each of the above to find results.

    3) msinfo32

    In left lower corner search type msinfo32 > wait 5 minutes for it to load > save to desktop > use share link to post into thread

    4) dxdiag

    In left lower corner search type dxdixg > wait 5 minutes for it to load > save to desktop > use share link to post into thread.

    Thursday, September 21, 2017 10:45 PM