none
PKI Client starts to intialize, then 7 minutes later client agents go back to disabled for upwards of an hour

    Question

  • Starting a new thread on this as I have done much digging and no longer believe its a conflicting record issue.

    I am SCCM 2012 SP1 on Server 2008 R2/SQL 2012 CU2.  My "Lan" based management point is HTTP and I also have an Internet Management point that is HTTPS.  All scenerios discussed in this thread are installing the 2012 SP1 client while connected to the Lan.  We have a fully functional PKI infrastructure with auto enrol enabled.

    The issue at hand:

    Wether I install the client during OSD, or manually install the client I get the same result.  This is that the client agent upon successful install begins communicating with the HTTP management point and starts retrieving policy.  If I open the ConfigMgr applet in Control Panel I see the client shows "Client Certificate: PKI", "Connection Type: Currently Intranet".  If I view actions I see all actions with the exception of Discovery Data Collection Cycle and Hardware Inventory Cycle.

    I have watched the client logs the only thing that seems to stick out is in the CcmNotificationAgent.log which shows the bgb client agent actions.  it repeats the following aprox every 5 minutes:

    bgb client agent is starting...
    bgb client agent is disabled
    TCP Listener is disabled
    bgbController main thread us started with settings: [bgb enable = 0], {tcp enable = 0} and {http enable = 0}.
    Wait 3600 seconds for even notification

    The ClientIDManagerStartup.log files shows:

    PopulateRegistrationHint: Using the Certificate selected by the current version of SCCM to set the hint. ClientIDManagerStartup 1/23/2013 5:00:51 PM 2100 (0x0834)
    CCMCreateAuthHeadersEx failed (0x80004005). ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    PopulateRegistrationHint failed (0x80004005), expected upon first start of non-upgrade client. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Finding certificate by issuer chain returned error 80092004 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Unable to find any Certificate based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Raising event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230052.204000+000";
     HRESULT = "0x87d00215";
     ProcessID = 2352;
     ThreadID = 2100;
    };
     ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Raising pending event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230052.204000+000";
     HRESULT = "0x87d00215";
     ProcessID = 2352;
     ThreadID = 2100;
    };
     ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    PKI Client Certificate matching SCCM certificate selection criteria is not available. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Generated a new Signing certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
    Generated a new Encryption certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
    Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    [RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    SID unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
    HWID unchanged ClientIDManagerStartup 1/23/2013 5:00:57 PM 2348 (0x092C)
    Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
    GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
    Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
    Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
     Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
     Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
    [RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
    [RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
    [RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
    RenewalTask: Executing renewal task. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    >>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Raising event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230143.106000+000";
     HRESULT = "0x00000000";
     ProcessID = 2352;
     ThreadID = 2620;
    };
     ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    RenewalTask: Certificate has changed, initiating a renewal. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Aborting any pending registration. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    Re-registration/renewal initiated. Restarting the service. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
    [----- SHUTDOWN -----] ClientIDManagerStartup 1/23/2013 5:01:44 PM 2100 (0x0834)
    [----- STARTUP -----] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Machine: W21599927256 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    OS Version: 6.1 Service Pack 1 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    SCCM Client Version: 5.00.7804.1000 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Client is set to use HTTPS when available. The current state is 448. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    >>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Raising event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230145.722000+000";
     HRESULT = "0x00000000";
     ProcessID = 3612;
     ThreadID = 3888;
    };
     ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Raising pending event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230145.722000+000";
     HRESULT = "0x00000000";
     ProcessID = 3612;
     ThreadID = 3888;
    };
     ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    'RDV' Identity store does not support backup. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    CCM Identity is in sync with Identity stores ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    >>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Raising event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230145.752000+000";
     HRESULT = "0x00000000";
     ProcessID = 3612;
     ThreadID = 3888;
    };
     ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Raising pending event:

    instance of CCM_ServiceHost_CertRetrieval_Status
    {
     ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
     DateTime = "20130123230145.752000+000";
     HRESULT = "0x00000000";
     ProcessID = 3612;
     ThreadID = 3888;
    };
     ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
    Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    [RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    SID unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
    HWID unchanged ClientIDManagerStartup 1/23/2013 5:01:49 PM 3928 (0x0F58)
    Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
    GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
    Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
    Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
     Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
     Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
     Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
    [RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
    [RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
    [RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
    [RegTask] - Client registration is pending. Sending confirmation request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)
    [RegTask] - Client is registered. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379. Approval status 2 ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)

    After almost 7 minutes exactly, the all the client agent actions besides Machine Policy Retrieval and UserPolicy Retrieval disappear (as if client policy was Reset)

    If I let the client sit for 45-minutes to an hour, everything starts working again and works fine from there on out. (cycling the SMS Agent service nor rebooting the machine makes it recover until the 45-min to an hour pass)

    My command line I am using to install the client is:

    ccmsetup.exe /mp:MYLANBASEDMP /UsePKICert /NOCRLCheck CCMHOSTNAME="myinternetmp.mydomain.com" SMSSITECODE=P10 SMSCACHESIZE=7000 FSP=MYLANBASEDMP CCMLOGMAXSIZE=1000000

    If I take out the /usePKICert, /NOCRLCheck and CCMHOSTNAME Entries, the client install and continues to function without issue.

    Anyone have any others ideas on where to troubleshoot this issue?  It would make more sense if the client NEVER worked after install.  Tearing my hair out trying to figure out why it starts to intialize, then reverts, then comes back online and works fine.  This happens at both my primary site MP as well as my secondary site/mp.  It happens on my standard Win7 image as well as Windows 8 test machines so I dont believe its a client OS issue.


    Wednesday, January 23, 2013 11:48 PM

All replies

  • I should add that this apears to only affect client installs where an object already exists in the SCCM database.  On the machine in reference above, I can uninstall and reinstall the client over and over and witness the same behavior,  if i delete the object from the SCCM Database, then install the client the problem does not happen.  the client comes online and stays online.  :-/
    Thursday, January 24, 2013 1:38 AM
  • More information:

    I noticed that Windows Management Framwork 3.0 (beta) was installed on the primary site server.  I uninstalled this version and installed the latest released version.  I was then able to reinstall the client on the machine above and it did not go dormant after 7 minutes and continued to work without issue (using the PKI stuff).  I then rebuilt this machine completely and the client once again did not exhibit the behavior and continued work successfully. 

    I then pushed the client install to a Windows Server 2012 box and once again the client came online and stayed online.

    Both of these clients are located at my secondary site server subnet using a proxy management point.

    I also have a VMware client thats in the boundaries of the Primary Site server.  I built the machine a few days ago to see if the same thing would happen in a different location and it did indeed go dormant after 7 minutes and after about an hour came back online.

    After my success against the secondary I rebuilt the VM in the Primary boundary and it still goes dormant after 7 minutes.  :-/

    I noticed on the 2 clients here that worked that the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 1}, {tcp enabled = 1}, {tcp port = 10123} and {http enabled = 1}.

    On the client that exhibits the behavior (the vm in the primary site boundaries) the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 0}, {tcp enabled = 0}, {tcp port = 0} and {http enabled = 0}.

    Can't say for sure that this is whats causing the behavior but it does appear to be consistent for client that "fail" and then recover.

    Anyone have any suggestions on where to look next?



    Thursday, January 24, 2013 5:14 PM
  • Just a bit more info...just reinstalled the client again on one of the secondary machines where the behavior didnt reappear earlier and its back to going dormant after 7 minutes.

    I have opened a case with Microsoft since I am getting nowhere fast.  if I find a resolution I will post back to the forums.

    I would however still LOVE to hear from anyone out there who has SCCM 2012 SP1 on wether they see this behavior or not..

    Thursday, January 24, 2013 7:49 PM
  • I've done some quick tests in my lab (http, not https) and do see the same behavior. I already contacted the product group but did not get any reply yet. Please update this thread if you got feedback from MS.

    Torsten Meringer | http://www.mssccmfaq.de

    Friday, January 25, 2013 7:57 AM
  • Thank you Torsten!  Very glad to hear its not just me (and that i have not gone mad).  I should be hearing from the CSS guy today so I'll certainly post back if/when we find the root cause.  Not sure about you but it "feels" like it has something to do with the new "Client Notification" system in SP1 (BGB Agent). 

    Thanks again for the reply and testing.

    Friday, January 25, 2013 2:35 PM
  • Just got off the phone with MS Support and as expected, the gentleman had never seen this behavior.  I captured 2 sets of logs from different machines and uploaded to them.  He is going to escelate the issue with the logs for analysis.  Likely wont hear back until Monday.
    Friday, January 25, 2013 6:50 PM
  • Just out of curiosity, if you check ccmeval.log is there any sort of client remediation taking place during this time?

    Saturday, January 26, 2013 12:33 AM
  • Hi Adam, CcmEval.log is not yet created at the time the issue appears. 
    Sunday, January 27, 2013 8:29 PM
  • William, thanks for posting this!  I am having the exact same issue!  It has been driving me nuts trying to figure it out.  I was just able to validate, as a workaround, I can delete the record from database and reinstall client and it does appear to resolve issue as best I can tell.  I've had tons of clients go "Inactive" (Disabled) just like yours.  I haven't been able to find a pattern to why some clients work and some don't, besides deleting records from database.  You found any other info yet?
    • Edited by bcehr Tuesday, January 29, 2013 2:59 PM
    Tuesday, January 29, 2013 2:52 PM
  • Hey bcehr, I am still waiting to hear back from microsoft support.  I sent them client logs as well as ran a diagnostic util on the primary site server and uploaded the results.  I did this on Friday.  I sent an email yesterday asking for a status and was told they are still working on it.  Hoping I hear back today.  :-/
    Tuesday, January 29, 2013 3:31 PM
  • Hey, ok sounds good.  Let us know what you find out.  I'm also working on issue from my side, if I can find anything else, I'll post back an update.
    Tuesday, January 29, 2013 3:35 PM
  • Just an update for anyone watching this thread..

    I sent an email yesterday and asked for an update on the issue from the support person and never got a response.  :-/  Just sent another email requesting an update.  So at its stands, day 5 with CSS and still nothing to go on yet.  I will traveling the rest of the week so hoping to have something by the time I return on Monday.

    Wednesday, January 30, 2013 2:56 PM
  • Just another update, finally got a response from the support tech who unfortunately is requesting more time.  So no closer to a resolution at this time.  I'll update again when i get more info.
    Monday, February 04, 2013 10:08 PM
  • William, thanks for keeping us in the loop.  I also wanted to provide a little more info.  I found that my secondary management servers, also have the same issue, but the workaround for them does not work, best I can tell, no way to fix those machines.  Are you seeing this too?
    Friday, February 08, 2013 1:21 PM
  • I do see it at both my primary site and at my secondary site.  I "thought" at one point that deleting the record from the database was the definitive workaround however have since then experienced the issue against a machine that was not already in the DB there really doesnt seem to be any consistency with why/when it happens.  I pushed 3 clients yesterday (upgrade to 2012 SP1 pushed via SCCM 2007) and 2 of 3 of then installed and stayed online, while the 3rd went dormat for about an hour and a half before coming back online.

    I have still yet to hear back from MS on my support case.  If I inquire I am unfortunately told "we here at PRO and PRE are affected by an increased influx of case volume so I will ask for some more time on this" so have no clue if/when we will actually work towards resolving this issue.  :-/

    Friday, February 08, 2013 2:51 PM
  • hello William,

    i found your thread because i have a similar problem (maybe even the exact same), with the error message "Finding certificate by issuer chain returned error 80092004". i have a similar setup (lan MP with http and another internet MP with https).

    when i do a client push install i see this in ClientIDManagerStartup.log:

    i tried already a lot to fix this thing, and i'm short before reinstalling the entire PKI/cert things from scratch. so far i have found one workaround:

    http://austrianalex.com/2012/10/system-center-configuration-manager-sccm-2012-client-pki-and-subordinate-ca-woes/

    this actually works for me - we have our root CA certificate distributed through active directory to our clients, so when i remove the cert from SCCM primary site properties / client computer communication / trusted root certification authorities, installation of the clients works again, like described in the link above.

    however, i am not sure if this is a correct "solution" because i now got errors on other parts of the infrastructure (e.g. pxe driver installation), which may be because i do not have a trusted root cert set in SCCM....

    i thought i share this here so maybe you want to try this on your end too.

    EDIT:

    after a lot of testing, this "solution" does not help me, as PXE no longer works without the trusted root CA cert set in site properties.

    Tuesday, February 12, 2013 12:16 PM
  • hello again, i have done a lot of analysis in the last couple of days and i am now confident that i have the exact same issue as the OP. i opened a microsoft ticket too.

    Thursday, February 14, 2013 11:40 AM
  • Thanks for the post back guys.  My case has been opened with MS Support for approaching a month now and unfortunately all I get back from the assigned engineer is that they are super busy and need more time.  I have escelated to our AM who I hope can spark a flame with the support team.  The only other hope I have is that they are delaying response because the product team is working on a hotfix.  Might be wishful thinking..

    This is a show stopper issue for us so we cannot move our migration forward until this is resolved.

    I'll post back if/when we ever start troubleshooting this issue with MS Support.

    Monday, February 18, 2013 3:36 PM
  • A couple of quick questions:

    Do you have a trusted root certification authority set up in the admin console?

    If you do have this set up, are your client certificates issued by the specified trusted root CA?

    Monday, February 18, 2013 8:42 PM
  • Hey Adam, yes we do have our Root CA configured and its the same CA that issues client certs.  Once the client comes back "online" it works perfectly fine using PKI.  We have succesfully tested the devices rolling over to the Internet MP when connected a network other than our domain network.  The problem only happens on intial client install (or reinstall).  Once its dies (for a lack of a better term) it lays dormant for awhile but then eventually comes online and everything works as expected and never goes dormant again.

    Thanks!

    Monday, February 18, 2013 9:14 PM
  • Any news on this bug?
    Monday, February 25, 2013 11:01 PM
  • Not yet.  I sent them a bunch of information on Client Settings last week.  Still waiting on MS Support who have escelated the issue internally. 
    Monday, February 25, 2013 11:20 PM
  • same here. no news from MS except that they are in delay.
    Tuesday, February 26, 2013 7:37 AM
  • update, a microsoft tech has been in touch with me yesterday, and we looked at the issue on our server for roughly 2 hours. no solution was found but they will try to replicate the issue internally and get back to me asap.
    Wednesday, February 27, 2013 9:46 AM
  • Thanks for the update Robert..
    Wednesday, February 27, 2013 2:34 PM
  • For those of you that are having this issue, what's the issuer name of the trusted root CA?
    Wednesday, March 20, 2013 12:16 AM
  • Adam, for me uses the default CA naming convention (domain-servername-CA); ie. CONTOSO-PKISRV-CA
    Wednesday, March 20, 2013 1:28 AM
  • CN = ABCD Root CA

    DC = domain

    DC = eu

    (domain.eu is our internal DNS domain, ABCD is the company name, no special characters)

    fyi, yesterday, microsoft tech has been on our server again, still no solution. i have requested this to be escalated to the dev team, to me it looks like a bug in sccm, especially after all that the MS tech has tried on our server.

    Wednesday, March 20, 2013 10:21 AM
  • Robert, if you have configured a trusted root CA in the admin console, the client will only use a certificate for client authentication that was issued by that CA. This may explain what you were observing.

    Wednesday, March 20, 2013 6:16 PM
  • On my side, I can confirm all my clients have a certificate from the CA that SCCM trusts.  Also, I can confirm its not a certificate issue, because if I uninstall the SCCM agent, delete the record from SCCM database, then reinstall client, it works.  However, this is very inconvenient because this scenario can happen to any client.
    Wednesday, March 20, 2013 6:28 PM
  • I can also confirm its not a trust issue with the cert.  The Root CA I have configured in SCCM is the same CA that hands out certs to clients.  As stated above, once the client "comes back" after letting it sit for a while it works fine from then on out.  This means it properly talks over HTTP to the Intranet MP/DP when on LAN and successfully flips over to HTTPS and talks to the Internet MP/DP when off LAN.

    Wednesday, March 20, 2013 6:50 PM
  • Thanks to both of you for your detailed reports on the issues you are experiencing. Unfortunately, I'm out of ideas at this time as to what may be happening and I hope that you'll get the assistance that need from CSS to get to the bottom of what's happening here.

    Thursday, March 21, 2013 6:53 PM
  • Thanks Adam.  Appreciate the effort.  :-)
    Thursday, March 21, 2013 7:10 PM
  • Any update on this guys?
    Monday, April 08, 2013 8:48 PM
  • Had a PFE remoted in last week digging through more server logs and have sent additional verbose client logs.  Still waiting :-/
    Tuesday, April 09, 2013 1:48 PM
  • nothing new.

    MS tech insisted that the problem must be our root certificate, so i should recreate it.

    i contacted our hosting partner, who created the certificate and is using it for MS lync, and we decided not to go ahead with this as lync is working fine and the cert is showing no sign of problems. ticket back to MS...

    Tuesday, April 09, 2013 2:00 PM
  • Our Root Cert comes from our internal CA which works fine for every other application we use it for.  Given the fact that this issue also happen if I do not provide the IBCM settings during install and install only as Intranet sure doesn't sound like a certificate issue to me, unless there is still cert checking stuff happening even when not using PKI for client communication?
    Tuesday, April 09, 2013 2:56 PM
  • Not to mention that when the client does finally come back online, it works fine from there on out, including via PKI across the internet..Seems that if the cert itself was the issue then the client would never work properly..
    Tuesday, April 09, 2013 2:58 PM
  • last update, microsoft asked me for a fresh set of logs, which i provided (http working state and https failed state). after checking through them, they again told me  that it must be a problem with the certs, so we should recreate them. i still cannot do this (root certs are in use by our lync infrastructure, all other certs i already recreated three times without effect), so finally management cancelled this project and i closed the ms ticket. bottom line, we can't use sccm 2012 for https/IBCM.

    not to hijack this thread for the wrong reasons, i posted here because i get the same error messages as the thread OP in the client logs:

    Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
    Finding certificate by issuer chain returned error 80092004 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834

    so our issue may have been for the same reasons, maybe not.
    i hope you find a solution for this, william, good luck!

    Tuesday, April 23, 2013 1:33 PM
  • Robert, in this specific case do you have a trusted root issuer configured in the admin console? If you do, the logging indicates that the trusted root issuer configured in the admin console hasn't issued a usable certificate on the client machine. See the earlier post I made on that piece of the issue and see if that helps you any.


    Check out my Configuration Manager blog at http://blogs.msdn.com/b/ameltzer

    Tuesday, April 23, 2013 9:12 PM
  • Hello Adam, yes we do (trusted root CA cert in console, as in your screenshot, and the client certs are issued by the same root CA through active directory enrollment)

    this root CA cert is the cause of our issue. once we remove the root CA cert from sccm console, clients immediately detect their machine certificate as valid and start communicating with the MP without the slightest delay. sadly we can't leave it with this, as pxe/osd now no longer works correctly. not going into too much detail here, we would need to fix the failure of sccm client to detect the machine cert as beeing valid one way or another, and MS tech's approach was to recreate the certs.

    we recreated all the client and IIS certs many times without success, strictly following the MS docs. i cannot recreate the root CA cert because too much production systems are depending on it (lync 2010) and apart from that, the certificate chain validates correctly from client cert to root CA cert so i don't see the issue here.

    bottom line, we were running in circles with MS tech support for over a month, and have now cancelled this project to move on with other things.

    Wednesday, April 24, 2013 6:33 AM
  • Hi

    Just want to inform you, and Microsoft if they are listening, this issue is not isolated to PKI environment. I havent got a PKI setup and i am experiencing the exact same issues. I have been battling with it for approx a month now and can find very little info out there about, unitl that was unitil i found this post.

    It is serously hindering my project, i am expecting to go 'LIVE' with SCCM based OS Deployment and then this starts happening, at first i thought it was becuase i was using ZTIWindowsUpdate.wsf as part of my task sequence so told everyone we cant use windows update, but i can see now that windowd update it not the cause.

    Could someone tell me how to log a call about this so i can also try to expediate things to get this resolved.

    Cheers..

    Thursday, April 25, 2013 4:25 PM
  • Hi Torsten,

    Im just wondering if you heard anything else regarding this ?

    Cheers

    Friday, April 26, 2013 11:03 AM
  • Hey Overfiend, I can tell you that my support engineer is still looking into it and have no additional updates at this time.   Going on 3 months now with having this case open with Microsoft with no movement.  :-/
    Friday, April 26, 2013 2:54 PM
  • Hi William,

    im sitting at a clinet right now and seeing this over and over again. The annoying thing is that its so random aswell, sometimes it 'appears' to be ok, then bang, it fails.

    This is not good, it renders the computer unusable for my purpose.

    I can't beleive its taken this long to fix..

    Have you applied CU1 yet ?

    Friday, April 26, 2013 2:57 PM
  • Yeah, we upgraded to CU1 this week.  No change for this issue..
    Friday, April 26, 2013 3:45 PM
  • Also seeing this same issue. Happens very randomly and as far as I can tell with no environment changes, or warnings for changes.

    2012 Configuration Manager only environment, client has a the new SP1 agent. Was working just fine and even though there was no changes all of the sudden says "Client is set to use HTTPS when available. The current state is 448" and PKI none.


    Monday, April 29, 2013 5:57 PM
  • Just another update to state we still have not received any movement from Microsoft on this issue.  Its been over 3 months now and I am rapidly approaching the point where I just punt SCCM 2012 and any potential Windows 8 tablet projects we have in the pipeline sadly and maybe try to revisit next year (assuming the issue has been identified and corrected by then of course).  This sucks.  :-)  Anyone else hear anything by chance from MS on this?

    Wednesday, May 01, 2013 2:02 PM
  • Hi William,

    I had some difficulty logging a  case with Microsoft so am looking here for answers and responses.

    For the past two days i havent had any failures, all seems to be well. I had made one slight change to my setup, i was getting a lot of errors in my SMS_INVENTORY_DATA_LOADER log, stating that mesages or inventory data couldnt be processed due to the MAX MIF Size being to small. The recommendation was to increase it larger than its default of 5MB (5000000), so i increased it to 20MB (20000000) just to get things working. Since then i havent had any errors. This could be just coincidence or good indeed be bad data getting stored in the database for the client machines expereinceing the problems..

    The reg key in question was ..HKLM\Software\Microsoft\SMS\Components\SMS_INVENTORY_DATA_LOADER, the key your looking for is Max MIF Size

    Let me know if this works for you, or anyone else.

    I only found this by looking at the errors being flagged up in my Sites Components Status, worked through trying to eliminate some of the errors that were appearing.

    Cheers.

    Wednesday, May 01, 2013 3:31 PM
  • Overfiend, are you saying you were having the issue of client installs coming online, then go offline for awhile and then come back on and start working (As is the basis for this thread) and when you increased your MIF size the client install issue disappeared?  Just want to make sure we are still talking about the same (original) issue.

    If its any consolation, I am not getting inventory processing errors in SMS_INVENTORY_DATA_LOADER so assuming this is not the same issue (for me at least).

    Thursday, May 02, 2013 5:40 PM
  • On my side, I still have the issue as well.  I noticed today, the issue typically comes back when I either update for SCCM Agent, or reinstall Agent.  If I ever do that, the agent gets Disabled like you have posted, and I have to uninstall, then delete the record from the Database, then reinstall from scratch.  Until someone finds a solution on this, I'm terrified to update to any CUs.  You see this same behavior?
    Friday, May 03, 2013 6:17 PM
  • I see this behavior on almost every install, whether its a new install or a reinstall.  In ALL cases however if I just let the machine sit, the client will come back online (typically about 45 minutes) and then work just fine from there on out.  Do your clients eventually recover bcehr or once they are disabled they never "self-correct"?
    Monday, May 06, 2013 1:32 PM
  • Best I can tell, they don't recover, they stay stuck disabled forever. I have to delete them from database and reinstall to get them to wake up.  The one way that agents do work correctly are ones that are installed via OSD, best I can tell, those agents work immediately and continue to work if untouched.  I don't do many "new" installs outside of OSD, so not sure if those might break too.
    Monday, May 06, 2013 1:35 PM
  • Hmm, well ok, that different then from what I am seeing.  :-/  On your side have you checked the conflicting records node to ensure there isn't an issue there?  They are supposed to auto merge for any domain joined clients who's hardware hasn't changed but might be worth a look just in case...
    Monday, May 06, 2013 9:36 PM
  • Hi.

    Just got word from Microsoft support that they have raised a hotfix for this problem. It's beeing tested out at some other customer now. They should get back to me in a couple of weeks.

    I'll keep up updated.

    Bjørn Erik

    Wednesday, May 22, 2013 7:50 PM
  • Seeing exactly the same issue here in a PKI only environment.  Root CA is configured in site properties and is also present on clients under Trust Root Certification Authorities.  Certificates issued to the SCCM environment, including server and client certificates are issues by subordinates.  Be keen to see a fix for this...as the OP said the client does come back to life within around an hour but this means that any machine rebuilt for a customer is missing key software for quite some time after build.

     
    Thursday, May 23, 2013 9:09 AM
  • I can also confirm that this has now been submitted to the Product Team as a bug and will be corrected with a hotfix or CU.  Now we just wait.  :-/
    Tuesday, May 28, 2013 9:47 PM
  • That's good news.  We are stuck until we have a resolution.  Any idea if this will be just a client hotfix?
    Wednesday, May 29, 2013 8:06 AM
  • Nothing official on what systems the hotfix will apply to.  If I were a betting man, I would bet its a server hotfix but only time will tell.
    Wednesday, May 29, 2013 1:13 PM
  • Just curious, but is this the hotfix you are referring to http://support.microsoft.com/kb/2841764 ?  it was just released yesterday.
    Thursday, May 30, 2013 1:45 PM
  • Unfortunately no.  This appears to fix an issue with having a 3rd party cert with the & symbol in it causing clients to not ever become assigned. 

    Thursday, May 30, 2013 6:33 PM
  • This fix makes some changes to code responsible for site assignment and HTTPS certificate publishing into AD. I would recommend trying it out to see if it manages to resolve any of your issues.

    It contains both client and server-side fixes.


    Check out my Configuration Manager blog at http://blogs.msdn.com/b/ameltzer

    Friday, May 31, 2013 7:52 PM
  • If anyone else tries this please post back your results.  I am going to hold out for the "official" hot fix for my particular issue from MS.  ;-)
    Monday, June 03, 2013 6:30 PM
  • Did MS give you any sort of ETA for hotfix or CU that will contain fix?
    Monday, June 03, 2013 6:34 PM
  • Unfortunately no.  I was simply told that the issue has been identified as a bug and that the product team was working on a fix that would be released sometime in the future as a hotfix or as part of a CU.  :-/
    Tuesday, June 04, 2013 1:44 PM
  • Any hotfix ? I've spent days and days on this and finally found this thread.

    MS support person working on this hasn't got anywhere yet.

    Either client registration works (clear the trust root cert in site client communication tab) and pxe fails, or the other way around.

    :(

    Thursday, June 27, 2013 2:44 PM
  • The last update I got was that this was identified as a bug and will be released with CU3.  No ETA on when CU3 will be released.  CU2 was just released a few days ago so I would think several weeks before CU3.    :-/
    Thursday, June 27, 2013 2:55 PM
  • William, I want to follow up internally to make sure this issue is being tracked properly. Could you let me know what the ticket number is for your incident?

    Check out my Configuration Manager blog at http://aka.ms/ameltzer

    Monday, July 01, 2013 10:46 PM
  • Hey Adam, the Incident number is "113012410165068".  The incident was "closed" several weeks ago however and I was told the resolution is that it's been identified and to wait for a fix.

    The information I gave above was from another MS employee who I met during a sessions at the local MS office here in STL who has been kind enough to dig around for information for me. 

    Tuesday, July 02, 2013 4:09 PM
  • Is there still no hotfix out for this?
    Monday, August 05, 2013 1:26 PM
  • Still nothing yet bcehr.  From the latest info I have its supposed to be included in CU3.  No eta that I am aware on a release date.
    Monday, August 05, 2013 4:21 PM
  • Just want to add I am having a remarkably similar issue (with the exception being that I am not running native mode).

    Currently, working with Microsoft on it and I even referenced this thread and your issue.

    He hasn't given me any additional info on CU3 either.

    Wednesday, August 28, 2013 1:09 PM
  • So CU3 has been released: http://support.microsoft.com/kb/2882125/en-us

    I don't see anything in the noted fixes that translates directly to this issue however we are going to proceed with it and keep our fingers crossed.  I'll post back when testing is completed.

    Monday, September 23, 2013 1:41 PM
  • I think this fix may be related (since I did observe this on my systems when the issue occurred):

    • The reimaging of an existing client computer may result in a policy being invalidated. This issue can occur if the SMS Agent Host service restarts before all Data Transfer Service (DTS) jobs are completely processed. Additionally, the PolicyAgent.log file on the reimaged client will contain entries that resemble the following:
      [CCM_Policy_Policy5.PolicyID="ScopeId_<var>GUID</var>/Application_<var>GUID</var>/CA",PolicySource="SMS:PRI",PolicyVersion="1.00"] is pointing to invalid DTS job [{<var>DTS_JOB_GUID</var>}]. Will attempt to re-download.

    Monday, September 23, 2013 2:54 PM
  • Yeah, that's the one item that gives me a glimmer of hope!
    Monday, September 23, 2013 3:44 PM
  • Unfortunately, from my initial testing, I am still seeing the issue.


    • Edited by WillMF Monday, September 23, 2013 9:20 PM
    Monday, September 23, 2013 9:19 PM
  • That's discouraging.. I will be testing tomorrow and will report back my findings.
    Monday, September 23, 2013 9:39 PM
  • I can confirm that CU3 DOES NOT fix this issue. Disappointing to say the least.  :-/  It has now been almost 9 months and still no fix. :(
    Tuesday, September 24, 2013 8:34 PM
  • Just to double-check and to avoid potential confusion: have you installed CU3 during the task sequence?

    Torsten Meringer | http://www.mssccmfaq.de

    Tuesday, September 24, 2013 8:38 PM
  • Hey Torsten,My testing came from uninstalling the client/reinstalling the client (with the CU3 msp using the PATCH parameter).  But to answer your question yes, I have updated my Task Sequences to point to CU3 during client install (replacing CU1 as we skipped CU2).

    I have been able to reproduce this issue regardless of how the client gets installed so its a safe assumption at this point that when I do test my updated TS that I will get the same result.

    Tuesday, September 24, 2013 9:18 PM
  • Just an FYI for anyone following this thread.  I have also update the escalation engineer at Microsoft with this information.  I'll post back if/when I get any additional info...
    Tuesday, September 24, 2013 9:20 PM
  • I am still testing to confirm, but I think I may have found the trigger for this issue (at least in my case).

    When testing, I have always immediately logged into the re-imaged system as soon as imaging completed so I could look for the symptoms of the issue (client works initially, then stops working, then starts working again after 45 minutes - hours later). In that case, there was a high likelyhood that I would experience the issue.

    I decided to re-image the systems and not login immediately after and see if anything changed. When I did this, I noticed (by monitoring the client logs) that (in four tests so far), I did not experience the issue.

    I have been trying to find a sweet spot as far as how long I should wait from the first client initialization after imaging before logging in (10 minutes, so far) without triggering the issue.

    Figured I would post to see if anyone could replicate the results and possibly pass this on to Microsoft.

    Thursday, September 26, 2013 8:10 PM
  • Hmm, I haven't tested this since rolling out CU3 however I am "almost" positive I did this same test several months ago and got the same result. Can't say for certain though so I'll give it another try and see. I'll actually test with uninstalling and reinstalling the client remotely while no one is logged in. I'll report back my findings.

    Thanks WillMF

    Friday, September 27, 2013 1:18 PM
  • WillMF, curious if you have a root cert configured on your primary site server?
    Friday, September 27, 2013 1:34 PM
  • No root cert.
    Friday, September 27, 2013 1:39 PM
  • Thanks for the quick reply. Well that blows my theory that it was only related to infrastructures that are using PKI. :-/ Don't understand why some (shall I say most) are not having (or at least not reporting) this issue. I was able to reproduce the issue in my lab as well. What the heck are we doing so different??!!
    Friday, September 27, 2013 2:15 PM
  • So my first test of reinstalling the client without a user being logged in, waiting for 15 minutes, then logging in did NOT reproduce the issue. Going to try it a few more times to ensure it wasn't an anomaly. Doesn't really fix anything obviously but good info to know..
    Friday, September 27, 2013 2:56 PM
  • Yeah, my lab setup as it is now is pretty much a standard OOB SCCM 2012 setup with a few desktops, a laptop, and a VM.

    Like you, I was surprised more people haven't seen/noticed this.

    So far, when not logging in immediately after re-image, the issue does not appear to be coming back.

    I am running through a few more tests on my "not logging in immediately" theory. Then I am going to run another test logging on immediately after re-image again and see if the issue returns.

    Friday, September 27, 2013 3:25 PM
  • I can confirm that when I install the client without a user being logged on and wait 10-15 minutes before logging on after the client is installed the issue does not appear. 

    For sanity check I logged on to the machine and then installed the client (same way as previous, using psexec remotely) the issue came right back.

    Definitely related to having a logged on user.  :-/

    Friday, September 27, 2013 4:15 PM
  • Hi,

    We rolled out CU3 because of the problems you talk about. We also have the same problems with approx. 10~15 clients a week.

    Waiting for a solution. Disappointed.

    -Dietmar-

    Monday, September 30, 2013 4:08 PM
  • Just checking in to see if you have received any update/s from the Microsoft escalation engineer you were working with.
    Monday, October 14, 2013 5:04 PM
  • Nope, honestly been busy and forgot until this post. I just sent another email. So far no response. :-/
    Monday, October 14, 2013 5:52 PM
  • Just got the following response:

    Hope you are doing well. I am really sorry for the late reply since I was out of office for last two weeks. However, I just check few of the documents and found that the fix is not available in CU3. Since it was not a business critical issue so it was not fixed in CU3 it will have a fix soon.

    Have a nice day.

    Monday, October 14, 2013 6:03 PM
  • Just a general FYI for the thread.

    I have updated my lab setup to SCCM 2012 R2 and this issue still occurs under the same scenarios previously discussed.

    Tuesday, October 22, 2013 6:50 PM
  • True. Just noticed that in my lab, too :-(

    Torsten Meringer | http://www.mssccmfaq.de

    Tuesday, October 22, 2013 8:07 PM
  • Thanks for the follow up gentlemen.  Still waiting for a fix :-|
    Wednesday, November 06, 2013 4:27 PM
  • This is my unhappy face -----> :-[

    I have this issue with a client and read this thread (thank you Torsten) with sad eyes.

    Non-native, 45 minutes +, Clients come back into shape and work thereafter ok.

    We are going to try and build and not login, thanks for the workaround ... MS fix it now this is annoying the business so it must be critical!


    Rob Marshall | UK | My Blog | WMUG | File CM12 Feedback | CM12 Docs | CM12 Release Notes

    Thursday, November 28, 2013 3:59 PM
    Moderator
  • Yeah, its been a LONG time since this issue was identified (as evidence of my original post date).  Can't believe its still not resolved.  There has been a SP, 3 CU's, and a new version.  :-/

    I contest its not business critical.  When I rebuild an existing machine the users may get notified of Available Software but by the time they go to install it, the client likely has gone quiet and presents them with a nice error message.

    Although I am certainly not happy others are having the same issue, maybe if enough people cry out about it they will get around to correcting it.

    Monday, December 02, 2013 2:42 PM
  • Definitely critical for me since I cannot proceed with anything but a lab deployment until this is resolved.

    I would open another case with Microsoft, but the last time I did that for this issue they were too busy focusing on trying to find things I was doing wrong and wasting my time.

    Friday, December 06, 2013 12:54 AM
  • So I received a call from Microsoft Support yesterday saying asking if this issue had been resolved stating that it was supposed to be fixed in CU3 (contrary to what my previous support tech told me).  As you might expect I informed him that the issue had not been resolved in CU3 (and that I didn't see anything in the "resolved" list from the CU3 KB article that stated this issue had been fixed).

    I was told I need to open a new case as this issue is now considered NEW due the fact that it should have been resolved in CU3.  :-/ 

    Unfortunately it appears I get to now rerun diagnostics and verbose debug logs again.  Good times.  I'll keep this thread posted with any progress.

    Friday, December 06, 2013 4:18 PM
  • Good news! (and potentially bad news)

    Good news, I received and email from the new support tech stating that indeed this issue was NOT fixed in CU3, however there is a workaround:

    At present we have a workaround for the issue by setting  the following registry

    HKLM\Software\Microsoft\CCM\UserPolicyReRequestDelay (REG_DWORD) value: 6,000,000 (decimal).

    Please add a step in your Task Sequence to add this registry value.

    If you are also facing the issue while trying to install the client manually then please follow these steps

    1. Install the client manually

    2. Immediately disable and stop the CCMExec service (SMS Agent Host)

    3. Set the following registry HKLM\Software\Microsoft\CCM\UserPolicyReRequestDelay (REG_DWORD) value: 6,000,000 (decimal)

    4. Enable and set the CCMExec service to automatic

    5. Start the CCMExec service

    I have tested when doing a manual client install and it works perfectly..IF you make sure to stop CCMEXEC directly after the client finishes installing as denoted in step 2 above.  I have every reason to believe that will will also work during a Task Sequence base don these results.  I'll be testing that soon.

    The bad news: If you are using Client Push, this workaround would not work since there isn't a way to wrap a script around the installer to perform the steps above.  Maybe you could add this value to all machines prior to client push using GPP's or a startup script?

    I don't currently use Client Push so its not a huge issue for me, I will just need to adjust my machine startup script to perform the steps but can see how this will still be an issue for others.

    Either way, its a step in the right direction.  Certainly tells me they have identified the issue and will hopefully be including a true fix in an upcoming hotfix or update.

    Thursday, January 02, 2014 4:47 PM
  • Just another quick update...adding the reg key prior to client push install doesn't work as ccmsetup.exe removes any existing keys prior to installing the client.  :-/ Not sure how to get around that one.. 
    Thursday, January 02, 2014 7:27 PM
  • Many thanks for this post William.

    I will be very much interested in your procedure for adding this via a task seqeunce. Im currently also doing some testing and will report my results.

    cheers

    Friday, January 03, 2014 1:45 PM
  • I added the following Run Command Line task as the first step in the State Restore phase. 

    reg add HKLM\Software\Microsoft\CCM /v UserPolicyReRequestDelay /t REG_DWORD /d 6000000

     Initial testing looks good.

    Friday, January 03, 2014 8:40 PM
  • Ok, indeed all does look well...

    I havent had a single instance of a faulty sccm cleint.

    One thing i have noticed is when i try to install software from the Application Catalog the install gets stuck at "Evaluting Requirements ...This may take a few minutes" However, if i reboot the machine or restart the SMS Agent Host Service, then  try again the install seems to go through fine..

    Which then of course make me ask the question, what exactly is this workaround doing...  obviously delaying 'User Policy ReRequest Delay'  but exactly what effect is this having and is this whats causing the issue i am seeing above..???

    But overall., the workaround is great, lets just hopes it sticks until an official fix is released.

    Edit.....

    Seems that if i leave the computer alone for a while everything works first time.. could just be a case of me not allowing the SCCM Client agent time to settle down.

    • Edited by The Overfiend Monday, January 06, 2014 3:49 PM update..
    Monday, January 06, 2014 1:26 PM
  • Glad to see they have at least ID'd the issue this time.

    I am curious to know did they give you any indication as far as what exactly this setting is doing?

    For example, would it delay the ability for the system to detect user-targeted deployments? And if so, for how long?

    In any case, happy to see that this may finally be resolved in upcoming hotfix/CU :-)

    Monday, January 06, 2014 6:41 PM
  • Adding the line you noted does appear to prevent the client from experiencing the previously noted issue.

    Unfortunately, so far, it appears to cause user-targeted deployments to not be evaluated.

    Previously, before applying the noted fix, the client would start deploying the user targeted deployments once the client issue resolved itself.

    With the fix in play and over an hour since the image completed, no user targeted deployments have run.

    I may play with lowering the DWORD value (assuming it equates to a timeframe) to see if that resolves this new issue.

    Has anyone else who has tested this seen this?


    • Edited by WillMF Tuesday, January 07, 2014 8:50 PM grammar
    Tuesday, January 07, 2014 8:40 PM
  • Just want to update on my additional testing and tweaking. Please see results below:

     

    DWORD Value: 6000000

    Initial Client Policy Rec'd: 2:21PM

    Start of User Targeted App Deployments: 4:07PM

    Duration: 1hr 46 minutes

     

    DWORD Value: 1500000

    Initial Client Policy Rec'd: 6:50PM

    Start of User Targeted App Deployments: 7:16PM

    Duration: 26 minutes

     

    DWORD Value: 1000000

    Initial Client Policy Rec'd: 12:58PM

    Start of User Targeted App Deployments: 1:16PM

    Duration: 18 minutes

     

    I am going to continue testing to see if it is consistent.


    • Edited by WillMF Wednesday, January 08, 2014 6:31 PM
    Wednesday, January 08, 2014 6:30 PM
  • Just wanted to give my final update from my testing.

    I settled on a DWORD value of 500000 (i.e. reg add HKLM\Software\Microsoft\CCM /v UserPolicyReRequestDelay /t REG_DWORD /d 500000)

    With that setting added during the task sequence as per previous guidance, I am seeing the application deployments starting between 9 - 13 minutes after initial client policy is received and no recurrence of the original client policy loss issue.

    Thanks again William for seeing this through to what appears to be a positive conclusion.

    Thursday, January 09, 2014 7:28 PM
  • Thanks for the additional testing and feedback WillMF.  Glad to see some movement on this issue finally.  Quite a nasty bug.  I did reach back out to the support tech asking for clarification on the key and got the following response:

    As per my understanding this registry key adds a delay to the user policy request after the SMS Agent Host Service starts thus preventing a conflict condition with the sms agent. However give me some time to consult my Technical support leads who provided the workaround to get more information.

    So our suspicions and your testing are indeed validated.

    Thanks!

    Friday, January 10, 2014 2:07 PM
  • Just a general FYI for the thread.

    I installed CU1 for SCCM 2012 R2 and then tested imaging without the above noted tweak in play and the issue returned.

    So, it appears that without the tweak in play, the issue still persists in CU1 for R2.

    Monday, March 31, 2014 5:54 PM
  • That's disappointing to say the least.  This issue must really have the devs stumped.
    Tuesday, April 01, 2014 2:09 PM
  • Have you any news about a fix?
    Wednesday, July 23, 2014 7:04 AM
  • Hi TWEESTY,

    I have unfortunately moved on from that position and into a consulting role.  I can say though that with the 3 SCCM projects I have done with SCCM 2012 R2 (up to CU1) I have not seen this issue at other clients so honestly forgot about it.  :-/

    Friday, July 25, 2014 1:43 PM
  • After encountering something similar, even if without PKI, I ended up here and here is my explanation for what happened.

    - I created an image by a task sequence which during the image creation installed the SCCM client and talked with the SCCM server.

    - After this "conversation", a new device appeared in my system which showed simply as "unknown", and which is separate from the other default unknown devices used for targeting systems not existing in the database (these are named x86 Unknown Computer and x64 Unknown Computer).

    - after sysprepping the image during the task sequence I went ahead with introducing the computer into my SCCM environment (included in the domain, triggered discovery, waited to appear).

    - initially it appeared in the list as a separate device from the "unknown" one, but after a while, after in started to receive policies and tried to apply an update deployment, suddenly went quiet and actions previously available in the SCCM Client interface dissapeared.

    - after this behavior happened, the "unknown" device disappeared from my list (my presumption is that it had the same GUIDs or hardware IDs and got merged)

    - Because of this merge, the assigned policies must have been reevaluated on the server and the ones already assigned to the client temporarily revoked as the SCCM client appeared to be the same as the "unknown" one.

    - after around one hour, the client started to work again.

    My conclusion is that maybe this happens because of this image that when created in the SCCM environment, it also creates this kind of shadow client, waiting to create a conflict with new computers imaged with the same GUIDs.

    Maybe deleting the SCCM cert from the image (ccmdelcert.exe) before doing the sysprep step in the task sequence might help.

    Thursday, December 11, 2014 10:16 PM
  • Hey Narcis, not to rain on your theory but the environment I was in, we created all images in standalone MDT so the SCCM Client never touched the core image.  :-/
    Thursday, December 11, 2014 10:36 PM
  • Just wanted to say thanks for this thread, i thought I was going crazy when I would see the client become active, and then revert to a newly deployed state where it would not function and re-download client policy. Implementing this reg hack now and testing...
    Friday, January 09, 2015 9:30 PM
  • I have this problem as well and will be trying the DWORD reg key hack shortly.
    Monday, January 26, 2015 3:58 PM
  • This reg key helped out a bunch.  I used a value of 500000.  Thank you.
    Monday, January 26, 2015 9:17 PM
  • Good deal.  Thanks for the reply Enten
    Tuesday, January 27, 2015 2:43 AM
  • I just wanted to add that I had a case open with MS concerning a massive failure when attempting to upgrade from SP1 to R2 and after finally upgrading my client was still showing the symptoms described in this thread.

    From another thread I posted I was directed to this one and when I brought it up to MS support they were pleased I found this solution on my own and were going to suggest the reg key to me, but I found the "resolution" beforehand.

    Very...sketchy...support...

    Either way it sounds like its a known bug, but they do not know of any official patch in development.  I asked for a communication if support finds out if a patch will be released, but I'm not expecting a call/email ever.

    Wednesday, January 28, 2015 3:17 PM
  • I am getting ready to build a new server at a new site. I am surprised they still haven't fixed this.
    Thursday, January 29, 2015 7:27 PM
  • The only good news WillMF is that it doesn't affect all installs of SCCM.  I have built, and/or worked with several infrastructures since I had this issue at a particular place and have (luckily maybe) not seen it again myself. The configurations all very different.

    If its a separate infrastructure, maybe it will work?  Or does the issue follow the network...  Hmm, has anyone built a 'different' infrastructure on the same network or vlan and had the issue disappear? 

    Also for good measure,

    2012 R2 CU4 was recently released.  The release notes do not list any fixes that appear to address the issue, however I would love to hear feedback from anyone having this issue who updates to CU4.  :-)


    Wednesday, February 04, 2015 2:30 AM
  • Bracken, still seems to be happening to us in 2012 R2 CU4 as well.
    Thursday, April 09, 2015 6:49 PM
  • Thanks for the feedback Mike.  Mind blowing this is still an issue!
    Friday, April 17, 2015 2:15 PM
  • Just wanted to let you know I have this issue on 2012 R2 CU4 as well.

    My task sequence steps:

    1. Copy CU4 patches to local drive

    2. Install Config Mgr client with PATCH command (PATCH="C:\windows\CCMHotfixes\configmgr2012ac-r2-kb3026739-i386.msp")

    3. Exit provisioning mode (REG ADD HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CCM\CcmExec /v ProvisioningMode /t REG_SZ /d false /f)

    4. Clear system excludes (part of provisioning mode) REG ADD HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CCM\CcmExec /v SystemTaskExcludes /t REG_SZ /d “” /f

    After this I have a step to install one piece of software and then reboot the machine. After the reboot the client takes 30 minutes to initialize. Once it initializes it installs the next software. Any subsequents reboots see the client taking 30 minutes to initialize. Strangely after 120 minutes the client wakes up and all is well.

    This issue only started occurring this past Sunday. It worked fine last week and no change was made to anything on the server on Friday and Saturday. I have a case open with Premier Support!

    Wednesday, June 10, 2015 11:39 AM
  • Thanks clintcdsss, let us know if you get any good feedback from Premier.
    Wednesday, June 10, 2015 4:33 PM
  • Premier Support advised me to include this in my task sequence:

    An enterprise hotfix rollup is available for Windows 7 SP1 and Windows Server 2008 R2 SP1

    https://support.microsoft.com/en-us/kb/2775511/en-us

    I included it in my reference machine, captured it and added the Operating system Image to my task sequence.

    The issue persists :(

    I have noticed that the issue is not consistent. e.g. On one machine today I ran the t/s and it finished in the estimated 60 minutes. I deleted the machine from AD and SCCM and ran the task sequence again. The issue then occurred again in this second task sequence.

    Thursday, June 11, 2015 9:19 AM
  • This must be a severe stumper for dev team.  Its been what, 2 and a half years?  For those having the issue, have you tried the workaround posted above (user policy reg key)?  It did work for "better" for me and some others as well.
    Friday, June 12, 2015 9:30 PM
  • OMG known issue. Can't wait to try the fix tomorrow!

    1. Disable both tasks with the command lines to exit the provisioning mode

    2. Add a run command line task immediately after Setup Windows and ConfigMgr task

    3. Set the following command line : sc.exe config ccmexec start= delayed-auto

    4. Following that, create a Restart Computer task.

    Monday, June 15, 2015 2:56 PM
  • Seems I might have found another nasty side effect of this bug.  Windows Embedded 8.1 Industry Pro...  When the client flushes, it triggers a servicing mode restart.  The client is in the dormant state but the restart command persists.  The results in the machine sitting in servicing mode indefinitely.  Absolutely frustrating.
    Thursday, September 03, 2015 2:13 PM