none
SCCM Client Upgrade - Triggers Advertisement Rerun

    Question

  • Here's my question for the forums:

    I am sure everyone always prefaces their questions with this, but we run a pretty complex SMS/SCCM environment at our corporation.  We are almost done with our SMS 2003 to SCCM upgrade.  We've successfully done an in-place upgrade of our servers and also migrated through backup/restore from old hardware to new 64-bit hardware.  Had a few "challenges," but everything is working great now!  So...now comes the point where we do the client upgrades....and here is where my question begins:

    When we do our inplace client upgrade (the client of course will stay assigned to the same site), the client appears to purge it's policy and reload it during the process.  Of course when it does this, it sees all of the policy as "new" and then reruns all advertisements that are currently active.  Now, as good SMS/SCCM administrators, most of our advertisements are set up for: "do not rerun" or were only set up with one mandatory advertisement time.  This saves a majority of our clients from rerunning installs or triggering MSI repairs.

    The problem is two fold...what about all those clients who failed installs last?  Are they destined to kick off all of those failed installs unless we go through and purge all of our active advertisements?  This then also poses another problem.  With cache's being purged and programs updated, clients will download the new versions of the software before they trigger the "success will not rerun."  Again, luckily we are saved from the rerunning, but with large packages this will tie up our network and cause wasted downloads.

    So it boils down to this:  Is there a way to tell the clients to NOT rerun advertisements during the client upgrade - or are we forced to expire and delete all of our advertisements prior to the client upgrade?

    Thanks in advance to the community!
    Friday, February 06, 2009 5:42 AM

Answers

  • So this answer has been a long time coming.  I have an answer from PSS, but it took quiet a bit of work as well as code debugging.  This issue has to do with a default timeout value for compiling policy on the SMS/SCCM client from the Requested to Actual.  The default time for this is 2 minutes.

    When the client performs a policy reset (either manually or during a client reinstall), the client requests and then begins downloading its policy.  In our case the policy was being downloaded by BITS.  The client begins downloading all of the policy, and at the end of the two minutes, merges the policies from the requested to the actual portion of WMI.  If there are any policies that have not finished or are still being downloaded, the process errors and all policies are then reloaded again.  If this "purge" happens, then all advertisements are treated as new (because the actual side of WMI has been cleaned out).  You can test or validate this yourself through the use of the policyspy tool.  In the tool, there is a feature called "Policy Reset."  Select this, wait for it to finish (just a second or two), and then launch a Request Machine Assignments/Policy Refresh.  What this does is clear out the requested policy and in theory does a full policy refresh (not a delta).  It downloades all of the policy (you can watch in the events section), and after two minutes it attempts to merge it into the current policies.

    For our environment, we are VERY heavy on client policy.  We have maxed out our software inventory rules at 64, we have 75 metering rules, we have several very long task sequences, and have about 30 persistent distributions for various tasks.  All said, after a policy reset, on most of our clients it takes about 3 minutes to download all of the policies with the full xml body.  With this in mind, this is over the 2 minute mark which means that if you reset the policy on a client - it will fail and purge the current policies and start over.  This was validated when we removed all of our current advertisments for a client - performed the upgrade and our policy was finishing its download in about 1 minutes and it did NOT exhibit the behavior.  It was the additional time it took to download the 30+ active advertisements.

    After weeks of working with PSS and escalation engineers, the finally determined that this was the issue.  It was not until they dug into the actual code with the engineers that they determined this was the root of our problem.  Part of the problem is that this two minute setting has existed since SMS 2003 and so there was a large assumption we had done something in our environment like "broken our WMI."  The way to fix this was to include a MOF file with the client install to change the setting from 2 to something a little more manageable (like 5 minutes).  We did this by creating a MOF file and including it to run right after the ccmsetup (and it changes the setting before the first full policy download occurs).

    I will show the WMI Class information below - but I will preface this with the fact that if you believe this is the problem, you need to contact PSS.  Changing this setting without Microsoft support's guidance may lead to an unsupported environment.  CCM client restart is required.

    #pragma namespace("\\\\.\\root\\ccm\\policy")
    class CCM_PolicyAgent_Configuration : CCM_Policy
    {
        uint32          PolicyTimeUntilUpdateActualConfig = 7;
    };

    Our reasoning for needing this may have to do with the amount of policy we have as well as other factors like network traffic or supersubnetting which negates some of the network detection of BITS.  It may also be our load (4000 clients on one MP/DP with 60 minute policy eval), but these are all in the "supported" numbers (http://technet.microsoft.com/en-us/library/bb680869.aspx).  If you find yourself in this scenario, I suggest that you test this scenario by removing all but a few advertisements from your client and then performing the upgrade or SCCM client reinstall.  See if it dumps all client settings and then reloads policy.  If you find this works - you are probably dealing with this timing issue and need to be VERY clear when sharing this issue with PSS.

    If you are not good with MOF files - you can view and change this setting through WBEMTEST with the following documentation:
    -Wbemtest (connect)
    -Connect to:  Root\CCM\Policy
    -Enum Classes (recursive)
    -Double Click CCM_PolicyAgent_Configuration
    -Property is:  PolicyTimeUntilUpdateActualConfig (default is 2)

    REMEMBER - CONTACT PSS BEFORE USING THIS FOR PRODUCTION.


    (Sorry about the extra clicks on the "answer" Torsten, I was having trouble with my screen size). ;)
    • Marked as answer by Beamer25 Thursday, May 07, 2009 9:51 PM
    • Edited by Beamer25 Thursday, May 07, 2009 9:53 PM
    Thursday, May 07, 2009 9:51 PM

All replies


  • Clients keep a history of which advert id & program name they have already executed.
    THis data is not purged with a client upgrade.

    Assuming that your advertisements id's have not changed, no re-runs should occur.


    "Everyone is an expert at something" Kim Oppalfens Configmgr expert for lack of any other expertise. http://www.scug.be/blogs/sccm
    Friday, February 06, 2009 2:00 PM
  • Kim,

    I will agree that the client keeps it's history, but this doesn't really help solve the root of the problem. I will agree with you that clients keep their package/program history in the registry which means that no re-runs will occur as long as the advertisements are set to "Rerun if Fail", the registry hive has not been deleted, or the client previously succeeded the install.

    It appears that when I run the ccmsetup to upgrade our clients, and you watch the policy and execmgr logs, you will see the client delete its policy set and then comment that it's client settings (like software distribution) are disabled.  Then it enables it's client agent settings a minute later (once it validates it is a valid client again) and downloads the full policy set.  At THIS point, it appears that our clients rerun all of their active advertisements. 

    Again, we are "saved" by our registry most of the time since the clients state "will not rerun" - but this is only AFTER they have downloaded new versions if necessary and only saves us if the client succeeded the install last time.  There are many situations where manual intervention was used to solve a failed distribution and you do not want that old MSI kicking off again.

    Hopefully this clarifies the question.  This may be the default behavior during an upgrade or client install and we just have to clean up our active persistent advertisements, but I really want to validate that this is not either a bug, or an option I can disable.

    Thanks again to the forums!  Any help or insight is appreciated.

    Friday, February 06, 2009 2:35 PM
  • I know that I have not had many responses on my thread - but I have been spending time troubleshooting this agree with the statement that this does not appear to be the "default" or "desired" behavior.

    It appears that some clients do not exhibit this experience.  After the client upgrade, their execmgr.log only shows:
    • Policy is updated for Program: XXX, Package: YYY, Advert: ZZZ
    For clients having the "rerun" behavior - they go through several more steps:
    • Policy is updated for Program: XXX, Package: YYY, Advert: ZZZ
    • Policy deleted for advertisement ZZZ package YYY program XXX
    • Policy arrived for parent package YYY program ZZZ
    • Mandatory execution requested for program XXX and advertisement ZZZ
    • Creating mandatory request for advert ZZZ, program XXX, package YYY

    I considered possibilities like overlapping boundaries since we do have multiple sites and a test system, but there were no overlapping boundaries.  Currently we are using SMSSITECODE=AUTO, and have tried both /service and /noservice and experienced the no rerun behavior.

    Newly created SMS 2003 clients (those created after we upgraded to SCCM) appear to be immune from the rerun along with certian older SMS clients - but we have been unable to come up with a pattern at this point.

    I know this is a lot of info - but I am hoping that now that this is an "anomoly" occurance, someone may have more infomration or some idea where this problem is coming from.  It feels like some client configuration or certificate based problem.  Thanks!

    Tuesday, February 10, 2009 4:27 AM
  • We had a similiar issue with advertisements re-running if there was problems with the client. In the end we changed all of our software collections to subselect querys based on the paticular piece of software being in add/remove programs.  so we would advertise to a collection, they would get the software then come out of the collection because they have the add/remove programs entry.  Then if there was any problems with the client they would not recieve the software again because it reads add/remove programs and does not add them to the collection.

    Not the answer you where looking for im sure, but it saves the problem occuring.
    Wednesday, February 11, 2009 5:47 PM
  • Thanks for your comments Keithgeo. 

    We use the subselect queries for quite a few of our distributions.  They are nice since you can almost run the advertisement until you have "0" left without the new software.  Unfortunately we have quite a few other attribute based distributions and software based distributions that keep us from getting rid of all of our advertisements for every client prior to the rollout.

    With a variety of computers and distributions, we really need to get a handle on what the core issue is since it may take us months to uprade our entire corporation (we have a large number of computers with unique maintanence schedules).

    I am welcome to more thoughts on the issue, we have engaged Premier Support to assist us since this is such an unusual issue.
    Friday, February 13, 2009 12:27 AM
  • Beamer25 said:

    I am welcome to more thoughts on the issue, we have engaged Premier Support to assist us since this is such an unusual issue.

    Please update this thread when you got a solution from PSS.
    • Marked as answer by Beamer25 Thursday, May 07, 2009 9:50 PM
    • Unmarked as answer by Beamer25 Thursday, May 07, 2009 9:50 PM
    • Marked as answer by Beamer25 Thursday, May 07, 2009 9:51 PM
    • Unmarked as answer by Beamer25 Thursday, May 07, 2009 9:51 PM
    Friday, February 13, 2009 8:06 AM
    Moderator
  • So this answer has been a long time coming.  I have an answer from PSS, but it took quiet a bit of work as well as code debugging.  This issue has to do with a default timeout value for compiling policy on the SMS/SCCM client from the Requested to Actual.  The default time for this is 2 minutes.

    When the client performs a policy reset (either manually or during a client reinstall), the client requests and then begins downloading its policy.  In our case the policy was being downloaded by BITS.  The client begins downloading all of the policy, and at the end of the two minutes, merges the policies from the requested to the actual portion of WMI.  If there are any policies that have not finished or are still being downloaded, the process errors and all policies are then reloaded again.  If this "purge" happens, then all advertisements are treated as new (because the actual side of WMI has been cleaned out).  You can test or validate this yourself through the use of the policyspy tool.  In the tool, there is a feature called "Policy Reset."  Select this, wait for it to finish (just a second or two), and then launch a Request Machine Assignments/Policy Refresh.  What this does is clear out the requested policy and in theory does a full policy refresh (not a delta).  It downloades all of the policy (you can watch in the events section), and after two minutes it attempts to merge it into the current policies.

    For our environment, we are VERY heavy on client policy.  We have maxed out our software inventory rules at 64, we have 75 metering rules, we have several very long task sequences, and have about 30 persistent distributions for various tasks.  All said, after a policy reset, on most of our clients it takes about 3 minutes to download all of the policies with the full xml body.  With this in mind, this is over the 2 minute mark which means that if you reset the policy on a client - it will fail and purge the current policies and start over.  This was validated when we removed all of our current advertisments for a client - performed the upgrade and our policy was finishing its download in about 1 minutes and it did NOT exhibit the behavior.  It was the additional time it took to download the 30+ active advertisements.

    After weeks of working with PSS and escalation engineers, the finally determined that this was the issue.  It was not until they dug into the actual code with the engineers that they determined this was the root of our problem.  Part of the problem is that this two minute setting has existed since SMS 2003 and so there was a large assumption we had done something in our environment like "broken our WMI."  The way to fix this was to include a MOF file with the client install to change the setting from 2 to something a little more manageable (like 5 minutes).  We did this by creating a MOF file and including it to run right after the ccmsetup (and it changes the setting before the first full policy download occurs).

    I will show the WMI Class information below - but I will preface this with the fact that if you believe this is the problem, you need to contact PSS.  Changing this setting without Microsoft support's guidance may lead to an unsupported environment.  CCM client restart is required.

    #pragma namespace("\\\\.\\root\\ccm\\policy")
    class CCM_PolicyAgent_Configuration : CCM_Policy
    {
        uint32          PolicyTimeUntilUpdateActualConfig = 7;
    };

    Our reasoning for needing this may have to do with the amount of policy we have as well as other factors like network traffic or supersubnetting which negates some of the network detection of BITS.  It may also be our load (4000 clients on one MP/DP with 60 minute policy eval), but these are all in the "supported" numbers (http://technet.microsoft.com/en-us/library/bb680869.aspx).  If you find yourself in this scenario, I suggest that you test this scenario by removing all but a few advertisements from your client and then performing the upgrade or SCCM client reinstall.  See if it dumps all client settings and then reloads policy.  If you find this works - you are probably dealing with this timing issue and need to be VERY clear when sharing this issue with PSS.

    If you are not good with MOF files - you can view and change this setting through WBEMTEST with the following documentation:
    -Wbemtest (connect)
    -Connect to:  Root\CCM\Policy
    -Enum Classes (recursive)
    -Double Click CCM_PolicyAgent_Configuration
    -Property is:  PolicyTimeUntilUpdateActualConfig (default is 2)

    REMEMBER - CONTACT PSS BEFORE USING THIS FOR PRODUCTION.


    (Sorry about the extra clicks on the "answer" Torsten, I was having trouble with my screen size). ;)
    • Marked as answer by Beamer25 Thursday, May 07, 2009 9:51 PM
    • Edited by Beamer25 Thursday, May 07, 2009 9:53 PM
    Thursday, May 07, 2009 9:51 PM