locked
Application not access with spoolsv and printisolationhost having large amounts of threads, trying to use process dumps RRS feed

  • Question

  • Every so often, I have an application which does not allow new logins.  I have taken process dumps for spoolsv, cpsvc and a single printisolationhosts  process, which have high thread counts of 763, 214 and 715.   Normal thread counts are 26, 60 and 5 or 6. 

    I am teaching myself to use windbg, process explorer and reading forum posts.   Any assistance is appreciated. 

    I  have screen shots for Spoolsv, PrintIsolationHost and Cpsvc,  Analyze Wait chain selections from resource monitor, which show they are waiting on spoolsv.

    From reading several posts, I have used Windbg to process the dumps finding locks and threads with "criticalsection" entries.  Using this information I have tried to find if there is a locked file causing this issue. 

    Microsoft (R) Windows Debugger Version 6.3.9600.17200 X86
    Copyright (c) Microsoft Corporation. All rights reserved.

    Loading Dump File [C:\spoolsv.DMP]
    User Mini Dump File with Full Memory: Only application data is available


    ************* Symbol Path validation summary **************
    Response                         Time (ms)     Location
    OK                                             C:\Symbols
    Symbol search path is: C:\Symbols
    Executable search path is:
    Windows 7 Version 7601 (Service Pack 1) MP (8 procs) Free x64
    Product: Server, suite: Enterprise TerminalServer
    Machine Name:
    Debug session time: Fri Jan 30 10:18:00.000 2015 (UTC - 5:00)
    System Uptime: 49 days 2:14:23.457
    Process Uptime: 3 days 18:21:30.000
    ................................................................
    ................................................................
    .......................................................
    Loading unloaded module list
    ................................................................
    *** ERROR: Symbol file could not be found.  Defaulted to export symbols for KERNELBASE.dll -
    ntdll!ZwWaitForSingleObject+0xa: 00000000`77c9135a c3              ret

    I used "!locks" and used the "lockcount" enty to trace the threads, but none had a "CriticalSection" entry

     ****  I used  ~*  kn to list all threads, and then used "find"  to search for "CriticalSection" and wrote down the thread references.

    0:000> ~739 kn
     # Child-SP          RetAddr           Call Site
    00 00000000`1c10f198 00000000`77c8e4e8 ntdll!ZwWaitForSingleObject+0xa
    01 00000000`1c10f1a0 00000000`77c8e3db ntdll!RtlpWaitOnCriticalSection+0xe8
    *** ERROR: Symbol file could not be found.  Defaulted to export symbols for localspl.dll -
    02 00000000`1c10f250 000007fe`ea910cd9 ntdll!RtlEnterCriticalSection+0xd1
    03 00000000`1c10f280 000007fe`ea8f058d localspl!SplDriverEvent+0x219
    04 00000000`1c10f2d0 000007fe`ea8f089d localspl!SplPowerEvent+0x2373d
    *** ERROR: Symbol file could not be found.  Defaulted to export symbols for win32spl.dll -
    05 00000000`1c10f310 000007fe`e4f1ac61 localspl!SplDeletePrinterWithJobs+0x179
    06 00000000`1c10f360 000007fe`e4eed70d win32spl!InitializePrintMonitor2+0x3d15
    07 00000000`1c10f390 000007fe`e4eefa7b win32spl!ProviderEntryW+0x5481
    08 00000000`1c10f400 000007fe`e4f1177b win32spl!ProviderEntryW+0x77ef
    09 00000000`1c10f490 000007fe`e4f19a25 win32spl!ProviderEntryW+0x294ef
    0a 00000000`1c10f500 000007fe`e4f19b40 win32spl!InitializePrintMonitor2+0x2ad9
    0b 00000000`1c10f550 000007fe`e4f12cce win32spl!InitializePrintMonitor2+0x2bf4
    0c 00000000`1c10f5e0 000007fe`e4ec8e0d win32spl!ProviderEntryW+0x2aa42
    0d 00000000`1c10f690 000007fe`e4ec8d57 win32spl+0x8e0d
    0e 00000000`1c10f6e0 000007fe`e4ec8a16 win32spl+0x8d57
    0f 00000000`1c10f710 00000000`77c563e5 win32spl+0x8a16
    10 00000000`1c10f740 00000000`77c60c26 ntdll!TppTimerpExecuteCallback+0x105
    *** ERROR: Symbol file could not be found.  Defaulted to export symbols for kernel32.dll -
    11 00000000`1c10f7a0 00000000`77b359ed ntdll!TppWorkerThread+0x5ff
    12 00000000`1c10faa0 00000000`77c6c521 kernel32!BaseThreadInitThunk+0xd
    13 00000000`1c10fad0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d


    0:000> u ntdll!rtlentercriticalsection
    ntdll!RtlEnterCriticalSection:
    00000000`77c92fc0 fff3            push    rbx
    00000000`77c92fc2 4883ec20        sub     rsp,20h
    00000000`77c92fc6 f00fba710800    lock btr dword ptr [rcx+8],0
    00000000`77c92fcc 488bd9          mov     rbx,rcx
    00000000`77c92fcf 0f83e9b1ffff    jae     ntdll!RtlEnterCriticalSection+0x31 (00000000`77c8e1be)
    00000000`77c92fd5 65488b042530000000 mov   rax,qword ptr gs:[30h]
    00000000`77c92fde 488b4848        mov     rcx,qword ptr [rax+48h]
    00000000`77c92fe2 c7430c01000000  mov     dword ptr [rbx+0Ch],1

    0:000> u ntdll!rtlpwaitoncriticalsection
    ntdll!RtlpWaitOnCriticalSection:
    00000000`77c8e400 48895c2420      mov     qword ptr [rsp+20h],rbx
    00000000`77c8e405 55              push    rbp
    00000000`77c8e406 56              push    rsi
    00000000`77c8e407 57              push    rdi
    00000000`77c8e408 4156            push    r14
    00000000`77c8e40a 4157            push    r15
    00000000`77c8e40c 4881ec80000000  sub     rsp,80h
    00000000`77c8e413 488d0576900e00  lea     rax,[ntdll!LdrpLoaderLock (00000000`77d77490)]

    0:000> !cs 77d77490
    -----------------------------------------
    Critical section   = 0x0000000077d77490 (ntdll!LdrpLoaderLock+0x0)
    DebugInfo          = 0x0000000077d77100
    NOT LOCKED
    LockSemaphore      = 0x3C4
    SpinCount          = 0x0000000000000000

    Friday, January 30, 2015 7:26 PM