Sign in
 
Home20122008 R2200820052000LibraryForums
SQL Server >
> 
> 
Data Collection Set 'Server Activity' Failure Waiting on a kernel object(s) failed. Inner Error ------------------> The handle is invalid.

Answered Data Collection Set 'Server Activity' Failure Waiting on a kernel object(s) failed. Inner Error ------------------> The handle is invalid.

  • Wednesday, November 28, 2012 1:06 PM
     
     
    Sign In to Vote
    0

    All of the system data collection sets have been running for 6+ months without any issues, couple of weeks ago the 'Server Activity' collection set failed to restart after a planned maintenance server reboot.

    When attempting to manually start the collection set it indicates "success", but after refreshing the collection set it showing as stopped. Viewing the logs for the Data Collection (Server Activity) I am getting the message:

    Waiting on a kernel object(s) failed. Inner Error ------------------>
    The handle is invalid.

    After some research into this really unhelpful message I came across one posting where this error has been found to occur in SP1. We are running SP2 which include the fix so this shouldn't be the problem (ref http://connect.microsoft.com/SQLServer/feedback/details/585210/sql-server-2008-data-collection-execmasterpackage-timeout).

    To see more detail on the root cause of the error I enabled the additional tracing for data collection (http://blogs.msdn.com/b/sqlagent/archive/2011/07/13/enabling-additional-tracing-for-data-collector.aspx). This provided an exception stack which indicated that the issue was thrown to what looks like an existing file, referring to a location (e:\) which doesn't exist in our system.

    DCEXEC!26d0!1b88!2012/11/28!11:17:25::    wmain _IN
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::    Command line: dcexec -c -s 2 -i "S160OLTP\OLTP" -m 0 -e 0x09C1635DD4C4C6478A145D679AFE718F_1_STOP
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Application::Init _IN
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Console code page (m_codePage): 437
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::      Application::ParseCommandLine _IN
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::      Application::ParseCommandLine _OUT:00000000
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Application::Init _OUT:00000000
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Application::Main _IN
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Started continuous mode collection on thread 12036
    DCEXEC!26d0!2f04!2012/11/28!11:17:25::      StartCachedCollectionWrapper _IN
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::      Waiting for collection to exit or stop event to be signaled
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::       StartCollection _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::       StartCollection(iSetId = 2, iCollectionMode = 0, pwszInstanceName = S160OLTP\OLTP)
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::        CDataCollectorController::ExecuteCollectionSet _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::        SQL Instance Name: OLTP
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::         CDataCollectorController::TryExecuteCollectionSetOnce _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::Initialize _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::Initialize _OUT:00000000
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::PopulateCollectorPackageParameters _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CacheWindow=1, CacheDirectory='I:\MSSQL10.OLTP\MSSQL\dccache'
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::PopulateCollectorPackageParameters _OUT:00000000
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::PopulateSetPackageParameters _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CollectionSetUID='{49268954-4FD4-4EB6-AA04-CD59D9BB5714}', ExecutionMode=0, IsSystem=-1, LoggingLevel=0, DaysUntilExpiration=14, ExitEventName=''
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CDataCollectorController::PopulateSetPackageParameters _OUT:00000000
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CreateControlEvents _IN
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25:: e        ERROR: Win32 Error: GetLastError=183, retCode=0x4, function CDataCollectorController::CreateControlEvents, line 470, file e:\sql10_katmai_t\sql\mpu\shared\dc\runtime\controller\src\datacollectorcontroller.cpp
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25:: e        ERROR: Failed to create kernel event for collection set: {49268954-4FD4-4EB6-AA04-CD59D9BB5714}. Inner Error ------------------>
    Cannot create a file when that file already exists.

    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::          CreateControlEvents _OUT:00000000
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25:: e       ERROR: DC_FAILED: function CDataCollectorController::TryExecuteCollectionSetOnce, line 350, file e:\sql10_katmai_t\sql\mpu\shared\dc\runtime\controller\src\datacollectorcontroller.cpp
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::         CDataCollectorController::TryExecuteCollectionSetOnce _OUT:0x000004
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::        Collection set execution failed, retrying in 5 seconds...
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::        Waiting for exit event or retry timer failed.
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25:: e      ERROR: Win32 Error: GetLastError=6, retCode=0x4, function CDataCollectorController::ExecuteCollectionSet, line 278, file e:\sql10_katmai_t\sql\mpu\shared\dc\runtime\controller\src\datacollectorcontroller.cpp
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25:: e      ERROR: Waiting on a kernel object(s) failed. Inner Error ------------------>
    The handle is invalid.

    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::        CDataCollectorController::ExecuteCollectionSet _OUT:0x000004
    DataCollectorController!26d0!2f04!2012/11/28!11:17:25::       StartCollection _OUT:00000000
    DCEXEC!26d0!2f04!2012/11/28!11:17:25::      StartCachedCollectionWrapper _OUT:0x000004
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Collection thread returned (00000001)
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Waiting for collection thread to exit
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Collection thread signaled
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::     Application::Main _OUT
    DCEXEC!26d0!1b88!2012/11/28!11:17:25::    wmain _OUT:0x000004

    I am now stumped as I haven't found anyone else experiencing this issue.

    Any ideas, please help??

     

All Replies

  • Monday, December 03, 2012 7:15 AM
    Moderator
     
     Answered
    Sign In to Vote
    1

    Hi ScottyB4444,

    A system can hang if a package running as part of data collector stops processing but does not exit and stays in this state. Most system hangs can be resolved by stopping and restarting a collection set. Please refer to details in the following link:
    http://technet.microsoft.com/en-us/library/bb677178(v=sql.100).aspx

    Thanks,
    Eileen

     

    Eileen Zhao
    TechNet Community Support

     
  • Wednesday, December 12, 2012 3:54 PM
     
      Has Code
    Sign In to Vote
    0

    Thanks for the pointer Eileen, problem solved.

    Can't believe I missed it, but after reading the last section on system hangs it occurred to me that I hadn't checked that all dcexec.exe processes were killed when I disable the data collection service.

    For anyone else experiencing this issue I identified the dcexec.exe processes using the following in cmd line:

    TASKLIST /SVC /FI "IMAGENAME EQ DCEXEC.EXE"

    On the default setup of the data collection there should be two dcexec.exe processes constantly running, one for "Server Activity" and another for "Query Statistics". On our server there were five running with the data collection disabled.

    After killing all of the processes and restarting the data collection problem solved!!

    Still need to understand why the processes didn't shut down cleanly when the server was restarted, but at least I know where to look now.


    • Edited by ScottyB4444 Wednesday, December 12, 2012 3:55 PM
    •