locked
Downstream Servers not fully synchronizing RRS feed

  • Question

  • We have had a WSUS 3.2.7600.256 Server up and running years without any real issues. It is running on a Windows 2008 R2 Server and it is patched up to Windows 8/2012 level.

    The WSUS Server is tied to an SCCM 2012 R2 server which is handling the list of Classification and Products but not being using to deploy updates to the enterprise, the only update deployment we leverage SCCM for is during image build and capture.

    We leverage GPO to assign the servers and clients the WSUS server and we use bits to throttle it during business hours.

    Our goal is to move to a GPO that is assigned per site, as such I have deployed some downstream servers which are setup with an SSL connection back to the Main WSUS Server.

    The downstream servers are working perfectly while synchronizing up to 99% then they stall and fail the sync with this error:

    SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

    at Microsoft.UpdateServices.DatabaseAccess.DBConnection.DrainObsoleteConnections(SqlException e)   at Microsoft.UpdateServices.DatabaseAccess.DBConnection.ExecuteReader() at Microsoft.UpdateServices.Internal.DataAccess.HideUpdatesForReplicaSync(String xmlUpdateIds) at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ProcessHiddenUpdates(Guid[] hiddenUpdates) at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync() at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)

    As a point of troubleshooting have disabled the firewall’s on the Downstream Server and the main WSUS server and well as attempted without SSL, none of this changed the outcome it still dies at 99%. I have currently re-enabled the firewalls and re-established the SSL setup.

    I have this issue currently replicate in our test lab, so I can fairly actively attempt things.

    All of the Downstream Servers (as well as the WSUS Server) are showing version: 3.2.7600.256

    I have tried some attempts to fix this, when I run wsusutil -checkhealth the result in the event log shows:

    Event 10022: The last catalog synchronization attempt was unsuccessful.

    If I uncheck "This is a replica" in the setup wizard then it works, obviously not the configuration I desire.

    I have also Re-indexed using this: http://technet.microsoft.com/en-us/library/dd939795(v=ws.10).aspx

    Any ideas?

    Thanks in advance,


    Portland Public Schools / Systems Administrator II





    • Edited by Adam Seitz Monday, April 28, 2014 9:51 PM
    Monday, April 28, 2014 5:16 PM

Answers

  • After I cleaned up all the rogue computers and a declining so many updates I am still seeing them when I close my eyes, 10,000+. 

    I finally got through the issue and here is what it took:

    * Verify the "Network service" has Full Control over the following locations: 
     - C:\Windows\Temp 
     - C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files 
     - <WSUS DRIVE>\WSUS 
    * Install sqlncli.msi and sqlcmdlntuils.msi on downstream server 
    * CD C:\Program Files\Microsoft SQL Server\100\Tools\Binn 
    * Copy Tool\WsusDBMaintenance.sql to that location 
    * Run (as administrator) sqlcmd -S np:\\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query -i .\WsusDBMaintenance.sql 
    * Launch Powershell (as administrator) 
    * Browse to Tools folder on WSUS drive 
    * Execute 15 day cleanup script 
    * Run the Server Cleanup Wizard in WSUS 
    * Execute 30 Day script to setit back to 30 days from 15. 
    * sync, hope it works!

    The SQL Command from here worked:
    http://technet.microsoft.com/en-us/library/dd939795(v=ws.10).aspx

    The 15/30 day scripts are out there on the net you can't miss them if you search. I also scheduled a weekly automated task cleanup by using this script with the recursive switch executed from my main WSUS server.
    http://gallery.technet.microsoft.com/scriptcenter/WSUS-Server-Cleanup-f33d2658


    Portland Public Schools / Systems Administrator II


    • Edited by Adam Seitz Wednesday, May 7, 2014 11:42 PM
    • Marked as answer by Adam Seitz Wednesday, May 7, 2014 11:42 PM
    Wednesday, May 7, 2014 11:40 PM

All replies

  • Hi,

    This thread may give us some clue.

    WSUS Downstream server not synchronizing - SQL Timeout

    http://social.technet.microsoft.com/Forums/en-US/c626d72f-fe52-4d63-a6e0-b89413e7a093/wsus-downstream-server-not-synchronizing-sql-timeout?forum=winserverwsus

    It’s quite long, in general, check physical size of SUSDB.mdf. Enable WSUS debug log and check the result.

    You can also try KB9094546, if you also get event 386 as described in the aticle.

    Hope this helps.

    Tuesday, April 29, 2014 8:08 AM
  • Since I originally posted this request for help, I have followed these instructions on this page:

    http://sccm.co/sccm/2012/09/downstream-wsus-server-sync-fails/

    But it did not not help.

    I have also tried to just have one of my new downstream servers get it's updates directly from MS and that will allow it to sync (as I originally noted) I took that one step further and had my new downstream servers point to that WSUS server all of them are capable of finishing the sync. So with that knowledge we can prove the issue lies in my master WSUS server.

    The thread you linked was very informational, but the only relevant infomration I could get from it was to run the WsusDebugTool, which I downloaded and attempted to run and it crashes, I tried copying it to system32 as some said that woud allow it work, but that didn't get it either.

    Is there some other way to dump out the config info?

    SUSDB.mdf = 3.72 GB

    In checking reg keys at this path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Update Services\Server\Setup

    I verified that the setups for the Main WSUS Server and the DownStream Servers is  identical except that the Master WSUS Server ddtabase is using hosted on a remote SQL cluster and all of downstream servers are using internal WSUS database. That should matter, right?

    Thanks!


    Portland Public Schools / Systems Administrator II





    • Edited by Adam Seitz Tuesday, April 29, 2014 9:19 PM
    Tuesday, April 29, 2014 5:46 PM
  • The WSUS Server is tied to an SCCM 2012 R2 server which is handling the list of Classification and Products

    but not being using to deploy updates to the enterprise

    One of these two statements is not correct. This scenario is impossible to achieve. Either Configuration Manager is configured to have a Software Update Point role, or it's not. If it's configured to have a SUP, the only way the CM clients are getting software updates is via Configuration Manager.

    SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

    Discussed weekly in this forum. Searching on the keyword "Timeout" will get you the gold mine.


    Lawrence Garvin, M.S., MCSA, MCITP:EA, MCDBA
    SolarWinds Head Geek
    Microsoft MVP - Software Packaging, Deployment & Servicing (2005-2014)
    My MVP Profile: http://mvp.microsoft.com/en-us/mvp/Lawrence%20R%20Garvin-32101
    http://www.solarwinds.com/gotmicrosoft
    The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.


    Tuesday, April 29, 2014 10:55 PM
  • I will continue looking through the posts, thanks!

    Portland Public Schools / Systems Administrator II

    Wednesday, April 30, 2014 5:46 PM
  • I will continue looking through the posts, thanks!

    In addition, you may find this blog series from PatchZone.org to be helpful.

    WSUS Timeout Errors: When and Why, Eliminating and Avoiding


    Lawrence Garvin, M.S., MCSA, MCITP:EA, MCDBA
    SolarWinds Head Geek
    Microsoft MVP - Software Packaging, Deployment & Servicing (2005-2014)
    My MVP Profile: http://mvp.microsoft.com/en-us/mvp/Lawrence%20R%20Garvin-32101
    http://www.solarwinds.com/gotmicrosoft
    The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

    Wednesday, April 30, 2014 10:35 PM
  • Hello,

    How is everything going? Is the issue resolved?

    Monday, May 5, 2014 1:31 AM
  • Hello,

    How is everything going? Is the issue resolved?

    Hey there! I have been out for about 4 days due to flu I am back at it now, unfortunately there are no TechNet articles that help me get over the flu! :)

    I am back looking at it today though.


    Portland Public Schools / Systems Administrator II

    Monday, May 5, 2014 3:51 PM
  • Hello,

    How is everything going? Is the issue resolved?

    So, a day later and I have learned that our WSUS implementation is in sad shape. The problems mostly occurred because we had some bad computer records in WSUS that were showing up as "unknown" for all the fields and "Never" as a last connected status. 

    This caused our updates to never get to 100% completion, I have removed those records and the vast majority of our updates now show 100%.

    There were also so many versions of operating systems that we don't even support and their language packs. It was a big over-sight on on our part. We have been letting it kind of run as "set it and forget it" type mentality because we are all too busy with all of the other work we are working on.

    So at this point the WSUS server is not yet replicating to the downstream servers, but I am not yet done troubleshooting it either. I hope to have it sorted out in next couple days.

    Thanks again for the links!


    Portland Public Schools / Systems Administrator II

    Tuesday, May 6, 2014 4:54 PM
  • After I cleaned up all the rogue computers and a declining so many updates I am still seeing them when I close my eyes, 10,000+. 

    I finally got through the issue and here is what it took:

    * Verify the "Network service" has Full Control over the following locations: 
     - C:\Windows\Temp 
     - C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files 
     - <WSUS DRIVE>\WSUS 
    * Install sqlncli.msi and sqlcmdlntuils.msi on downstream server 
    * CD C:\Program Files\Microsoft SQL Server\100\Tools\Binn 
    * Copy Tool\WsusDBMaintenance.sql to that location 
    * Run (as administrator) sqlcmd -S np:\\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query -i .\WsusDBMaintenance.sql 
    * Launch Powershell (as administrator) 
    * Browse to Tools folder on WSUS drive 
    * Execute 15 day cleanup script 
    * Run the Server Cleanup Wizard in WSUS 
    * Execute 30 Day script to setit back to 30 days from 15. 
    * sync, hope it works!

    The SQL Command from here worked:
    http://technet.microsoft.com/en-us/library/dd939795(v=ws.10).aspx

    The 15/30 day scripts are out there on the net you can't miss them if you search. I also scheduled a weekly automated task cleanup by using this script with the recursive switch executed from my main WSUS server.
    http://gallery.technet.microsoft.com/scriptcenter/WSUS-Server-Cleanup-f33d2658


    Portland Public Schools / Systems Administrator II


    • Edited by Adam Seitz Wednesday, May 7, 2014 11:42 PM
    • Marked as answer by Adam Seitz Wednesday, May 7, 2014 11:42 PM
    Wednesday, May 7, 2014 11:40 PM
  • Thanks for sharing, Adam.

    I was not aware that the RevisionDeletion thresholds were even configurable via the API. I knew that the ComputerDeletion threshold is (we enabled that functionality in SolarWinds Patch Manager's instance of the Server Cleanup Wizard).


    Lawrence Garvin, M.S., MCSA, MCITP:EA, MCDBA
    SolarWinds Head Geek
    Microsoft MVP - Software Packaging, Deployment & Servicing (2005-2014)
    My MVP Profile: http://mvp.microsoft.com/en-us/mvp/Lawrence%20R%20Garvin-32101
    http://www.solarwinds.com/gotmicrosoft
    The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

    Thursday, May 8, 2014 4:18 PM
  • Thanks for sharing, Adam.

    I was not aware that the RevisionDeletion thresholds were even configurable via the API. I knew that the ComputerDeletion threshold is (we enabled that functionality in SolarWinds Patch Manager's instance of the Server Cleanup Wizard).


    Lawrence Garvin, M.S., MCSA, MCITP:EA, MCDBA
    SolarWinds Head Geek
    Microsoft MVP - Software Packaging, Deployment & Servicing (2005-2014)
    My MVP Profile: http://mvp.microsoft.com/en-us/mvp/Lawrence%20R%20Garvin-32101
    http://www.solarwinds.com/gotmicrosoft
    The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

    Well there is proof that we all learn something new every day! :)

    Thanks again for your links and advice.


    Portland Public Schools / Systems Administrator II

    Thursday, May 8, 2014 5:25 PM