locked
High DPC Latency caused by IopTimerDispatch in ntoskrnl.exe RRS feed

  • General discussion

  • Windows 7 64bit installed.
    The system shows a typical DPC latency of about 250µsec. (A Windows XP 32bit on the same hardware has typical DPC latency of about 50µsec!) The high latency is considered causing dropouts in realtime audio processing. All unnecessary drivers/hardware are already disabled in Device Manager (including network).

    xperf analysis shows that most of the  250µsec are spent in routine IopTimerDispatch in ntoskrnl.exe.
    IopTimerDispatch seems to be called about once every second (timer driven) and then consumes 200µsec CPU on average.

    I could not find anything about IopTimerDispatch.
    What is this function for?
    Why is it called?
    Why is it using excessive amounts of CPU time, violating Microsoft's own standards for DPC routines?

    I consider this a bug.
    Is there any way to report such bugs to Microsoft?

    • Changed type Juke Chou Thursday, February 24, 2011 8:01 AM
    Thursday, February 10, 2011 11:54 PM

All replies

  • VOID
    IopTimerDispatch(
      IN PKDPC Dpc,
      IN PVOID DeferredContext,
      IN PVOID SystemArgument1,
      IN PVOID SystemArgument2
      )
    
    /*++
    
    Routine Description:
    
      This routine scans the I/O system timer database and invokes each driver
      that has enabled a timer in the list, once every second.
    
    Arguments:
    
      Dpc - Supplies a pointer to a control object of type DPC.
    
      DeferredContext - Optional deferred context; not used.
    
      SystemArgument1 - Optional argument 1; not used.
    
      SystemArgument2 - Optional argument 2; not used.
    
    Return Value:
    
      None.
    
    --*/
    
    Which other drivers have a high latency?

     


    "A programmer is just a tool which converts caffeine into code" CLIP- Stellvertreter http://www.winvistaside.de/
    Friday, February 11, 2011 3:11 PM
  • I will add an overview csv extract of a 10sec interval from xperf below.
    The only other drivers showing >20µsec CPU usage are
    nvlddmkm.sys (Nvidia graphics driver) and
    dxgkrnl.sys (DirectX graphics driver).
    I consider all other drivers to be harmless.
    Note that I have disabled all non-vital hardware (including network). A driver that caused notable latency problems was ACPI.sys (Microsoft), which regularly used another 200µsec. I disabled it. It is no longer there.
    I also disabled Aero, as this caused huge DPC latency spikes of >15000µsec once every 30 seconds or so.

    I do not quite understand how the other drivers shown would be logically linked to IopTimerDispatch.
    If the other drivers would be invoked by IopTimerDispatch AND would show in the table, then their CPU usage would not be accounted for in IopTimerDispatch. That would be an indication that IopTimerDispatch itself has a problem.
    On the other hand, if the other driver's CPU usage would be accounted an behalf of IopTimerDispatch, then they would not show here. There is definitly no double accounting, because an external DPC latency checker confirms the regular 250µsec DPC latencies.

    It would probably be more interesting to find out what the contents of the I/O System Timer Database are and what drivers are hooked in there.

     

    DPC Type, Module, Function, CPU, Count, Max Actual Duration (ms), Avg Actual Duration (ms), Actual Duration (ms), % Actual Duration, Enter Time (s), Exit Time (s), Max Duration (ms), Avg Duration (ms), Duration (ms), % Duration
    Timer, , , , 103, 0.227373, 0.009069, 0.934116, 0.05, 5.009714891, 5.992495184, 0.230452, 0.009178, 0.945407, 0.05
    , ntoskrnl.exe, , , 70, 0.227373, 0.010250, 0.717527, 0.04, 5.009714891, 5.992495184, 0.230452, 0.010411, 0.728818, 0.04
    , , IopTimerDispatch, 0, 1, 0.227373, 0.227373, 0.227373, 0.01, 5.384165259, 5.384395711, 0.230452, 0.230452, 0.230452, 0.01
    , , PpmCheckStart, 0, 64, 0.013858, 0.007466, 0.477836, 0.02, 5.009714891, 5.992495184, 0.013858, 0.007594, 0.486048, 0.02
    , , KiScanReadyQueues, 0, 5, 0.005645, 0.002463, 0.012318, 0.00, 5.181338661, 5.976911220, 0.005645, 0.002463, 0.012318, 0.00
    , nvlddmkm.sys, , , 11, 0.038494, 0.006112, 0.067234, 0.00, 5.072126708, 5.976908141, 0.038494, 0.006112, 0.067234, 0.00
    , , 0xfffff88004bf6d80, 0, 1, 0.038494, 0.038494, 0.038494, 0.00, 5.384125226, 5.384163720, 0.038494, 0.038494, 0.038494, 0.00
    , , 0xfffff88004b32250, 0, 10, 0.003592, 0.002874, 0.028740, 0.00, 5.072126708, 5.976908141, 0.003592, 0.002874, 0.028740, 0.00
    , tcpip.sys, , , 4, 0.015398, 0.013601, 0.054406, 0.00, 5.025320925, 5.820912988, 0.015398, 0.013601, 0.054406, 0.00
    , , TcpPeriodicTimeoutHandler, 0, 2, 0.015398, 0.014628, 0.029256, 0.00, 5.306126879, 5.820912988, 0.015398, 0.014628, 0.029256, 0.00
    , , IppTimeout, 0, 2, 0.014371, 0.012575, 0.025150, 0.00, 5.025320925, 5.524524097, 0.014371, 0.012575, 0.025150, 0.00
    , NETIO.SYS, WfpSysTimerCallback, 0, 5, 0.012318, 0.006980, 0.034900, 0.00, 5.181341741, 5.477730119, 0.012318, 0.006980, 0.034900, 0.00
    , afd.sys, AfdTimerWheelHandler, 0, 1, 0.009239, 0.009239, 0.009239, 0.00, 5.883320185, 5.883329424, 0.009239, 0.009239, 0.009239, 0.00
    , rdbss.sys, RxTimerDispatch, 0, 1, 0.009239, 0.009239, 0.009239, 0.00, 5.384397764, 5.384407003, 0.009239, 0.009239, 0.009239, 0.00
    , ataport.SYS, IdePortTickHandler, 0, 2, 0.007185, 0.006159, 0.012318, 0.00, 5.352901353, 5.852135833, 0.007185, 0.006159, 0.012318, 0.00
    , tunnel.sys, TunnelTimeoutRoutine, 0, 2, 0.005646, 0.005132, 0.010265, 0.00, 5.384117527, 5.883318132, 0.005646, 0.005132, 0.010265, 0.00
    , iaStor.sys, 0xfffff8800109e6dc, 0, 1, 0.004619, 0.004619, 0.004619, 0.00, 5.742888979, 5.742893598, 0.004619, 0.004619, 0.004619, 0.00
    , luafv.sys, ScavengerTimerRoutine, 0, 1, 0.004619, 0.004619, 0.004619, 0.00, 5.384411622, 5.384416241, 0.004619, 0.004619, 0.004619, 0.00
    , ndis.sys, ndisMWakeUpDpcX, 0, 5, 0.002566, 0.001950, 0.009750, 0.00, 5.118897589, 5.118914013, 0.002566, 0.001950, 0.009750, 0.00
    Regular, , , , 201, 0.069290, 0.004994, 1.003924, 0.05, 5.009728749, 5.992895523, 0.069290, 0.005004, 1.005977, 0.05
    , dxgkrnl.sys, 0xfffff88003eed588, , 5, 0.069290, 0.026792, 0.133961, 0.01, 5.174296288, 5.992895523, 0.069290, 0.026792, 0.133961, 0.01
    , iaStor.sys, 0xfffff880010c0448, 1, 1, 0.016424, 0.016424, 0.016424, 0.00, 5.182950282, 5.182966706, 0.016424, 0.016424, 0.016424, 0.00
    , i8042prt.sys, I8042KeyboardIsrDpc, , 4, 0.014371, 0.010906, 0.043626, 0.00, 5.173890304, 5.414239440, 0.016424, 0.011419, 0.045679, 0.00
    , USBPORT.SYS, , , 74, 0.013858, 0.004952, 0.366462, 0.02, 5.026508084, 5.986521408, 0.013858, 0.004952, 0.366462, 0.02
    , , USBPORT_IsrDpc, , 37, 0.013858, 0.007768, 0.287422, 0.01, 5.026508084, 5.986517815, 0.013858, 0.007768, 0.287422, 0.01
    , , USBPORT_Xdpc_Worker, , 37, 0.002567, 0.002136, 0.079040, 0.00, 5.026518862, 5.986521408, 0.002567, 0.002136, 0.079040, 0.00
    , ntoskrnl.exe, , , 117, 0.011291, 0.003790, 0.443451, 0.02, 5.009728749, 5.992505449, 0.011291, 0.003790, 0.443451, 0.02
    , , PpmPerfAction, 1, 58, 0.011291, 0.005061, 0.293578, 0.01, 5.009728749, 5.992502369, 0.011291, 0.005061, 0.293578, 0.01
    , , PpmCheckRun, 1, 58, 0.009752, 0.002504, 0.145254, 0.01, 5.009736961, 5.992505449, 0.009752, 0.002504, 0.145254, 0.01
    , , EtwpFlushLoggerDpc, 0, 1, 0.004619, 0.004619, 0.004619, 0.00, 5.181378695, 5.181383314, 0.004619, 0.004619, 0.004619, 0.00

    Friday, February 11, 2011 8:20 PM
  • Have you installed the following hotfix?

    2494666        Stop errors may occur after you use the Driver Verifier manager to apply standard settings to all drivers in Windows 7 or in Windows Server 2008 R2

    http://support.microsoft.com/default.aspx?scid=kb;en-US;2494666

     

    which contains updates like the ones below:

     

    982383        You encounter a decrease in I/O performance under a heavy disk I/O load on a Windows Server 2008 R2-based or Windows 7-based computer

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;982383         

     

    2155311        Poor performance occurs on a computer that has NUMA-based processors and that is running Windows Server 2008 R2 or Windows 7 if a thread requests lots of memory that is within the first 4 GB of memory

    http://support.microsoft.com/default.aspx?scid=kb;en-US;2155311      

     

    With the exception of security updates, Microsoft hotfix updates are cumulative.  If you install the most recent Ntoskrnl.exe you will also get all the other updates/improvements that have been made.  2494666 is one of the most recent ntoskrnl.exe currently available.  Note: The version number may be confusing because there are two version numbers that appear to be identical at first glance.  There is a version that only includes security changes and a hotfix version with all the previous updates.  When updates are installed Windows detects if you had a security only version or a hotfix version.    If a hotfix had previously been installed then Windows installs the hotfix version even if you are installing a Security update to ensure the hotfix changes are preserved.  

     

    Saturday, February 19, 2011 11:55 AM
  • Trying to download the fix for 2494666, the website does not offer me a x64-version of the fix (only x86 and IA64). The article for 2494666 does mention the x64-version, but it does not appear as an option on the download page.
    Monday, February 21, 2011 12:11 AM
  • Hello nobodysomeday,

    Your question falls into the paid support category which requires a more in-depth level of support.  Please visit the below link to see the various paid support options that are available to better meet your needs. http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone

    Regards,
    Clark Satter
    Microsoft Online Community Support

     

    Thursday, February 24, 2011 12:23 AM
  • Windows 7 64bit installed.
    The system shows a typical DPC latency of about 250µsec. (A Windows XP 32bit on the same hardware has typical DPC latency of about 50µsec!) The high latency is considered causing dropouts in realtime audio processing. All unnecessary drivers/hardware are already disabled in Device Manager (including network).

    xperf analysis shows that most of the  250µsec are spent in routine IopTimerDispatch in ntoskrnl.exe.
    IopTimerDispatch seems to be called about once every second (timer driven) and then consumes 200µsec CPU on average.

    I could not find anything about IopTimerDispatch.
    What is this function for?
    Why is it called?
    Why is it using excessive amounts of CPU time, violating Microsoft's own standards for DPC routines?

    I consider this a bug.
    Is there any way to report such bugs to Microsoft?

    using latencymon i too get the high dpc in ntoskrnl.exe, wonder why things like this never get fixed...



    Say no to useless services.

    Tuesday, May 21, 2013 10:35 AM