Answered by:
Windows Update client (wuauserv) high CPU utilization

Question
-
For some reason this morning, it appears that the Windows Update client is causing the CPU to spike on all of my Windows Server 2012 (non-R2) servers. I looked closer at one server to troubleshoot. It was patched Friday night with the 2018-05 updates, but the issue only started this morning. Other than that nothing else has changed as far as I know. All of our servers use an internal WSUS server, and no other computers other than Server 2012 seem to be having this issue. I tried recreating the Software Distribution folder (stop wuauserv, rename folder, start), but that did not help. I have a custom script that calls the Windows Update API, and as soon it executes the Search method, the CPU goes up and the search never produces results. Any suggestions?
Answers
-
Manually updating definitions for SCEP has fixed it for me on one of the servers. I didn't have re-install SCEP. So, I guess, now it skips the update process from WSUS somehow, right? I wonder what happens when new definitions come out.
All replies
-
We are seeing a similar spike that started this weekend, only we did not patch our Windows 2012 (non-R2) servers. So this may not be the patch itself. My current guess is an update detection changed, causing these machines to have a problem.
Graph showing roughly when the spikes started here for us is attached.
-
I found that not all of our 2012 servers are experiencing this issue.
When I attempt to check for updates on one of these servers, it does not even appear to build a TCP connection to the WSUS server (run netstat while checking for updates). The WindowsUpdate.log shows the local WSUS server in the policy, but I don't see any events indicating that it is trying to connect. I have a couple Warnings, but nothing that looks to critical as far as I know.
WARNING: Network Cost is assumed to be not supported as something failed with trying to get handles to wcmapi.dll
WARNING: Failed to get Wu Exemption info from NLM, assuming not exempt, error = 0x80240037
EDIT: I also just tried hitting the "Check online for updates" which should send it out to Microsoft update servers. It does not appear to even make a connection either.- Edited by Brandon.M Monday, May 14, 2018 8:25 PM
-
-
I started a similar thread because I didn't see yours at the time. We're having the same issue with pretty much all of our 2012 non-R2 servers and it started yesterday/today with the detection routines.
I wonder if there was a change on the Microsoft side (such as a change to TLS 1.2, etc. on Windows Update server) that would cause some issue with 2012 servers? Our servers have a mix of April 2018 and May 2018 cumulative updates and regardless of which patch level they have they are experiencing the issue.
Our 2012 R2 and 2016 systems are not having any issues. I have not looked at the few 2008 R2 servers yet. I will have to do so tomorrow just to check...
-
-
-
-
1) Reinstall SCEP (FEP in my cause)2) create c:\temp
2) Invoke-WebRequest -Uri "http://go.microsoft.com/fwlink/?linkid=87341&clcid=0x409" -OutFile c:\temp\mpam-feX64.exe
3) cd c:\temp
4) ./mpam-feX64.exe -q
Alex
- Edited by ArhangeL87 Wednesday, May 16, 2018 3:46 AM
- Proposed as answer by ArhangeL87 Wednesday, May 16, 2018 3:46 AM
-
We are having the same issue. We don't have many 2012 (non r2) servers in our environment. The one we have that is experiencing the issue is virtual (esxi 6.0) and it is the WSUS server for us.
When this started it had 1 cpu and 4GB of memory. I doubled both. Now it sits at 50% cpu instead of 100%.
It is running SCEP and killing it did not help any. Only killing the wuauserv seems to help.
When this issue started, we had not run windows update on it since the April updates.
-Ben
-
Manually updating definitions for SCEP has fixed it for me on one of the servers. I didn't have re-install SCEP. So, I guess, now it skips the update process from WSUS somehow, right? I wonder what happens when new definitions come out.
-
The one server I have been troubleshooting on apparently updated its Definitions earlier this morning. Weird thing is that I disabled Windows Update service yesterday and it was still disabled when I just checked it. Maybe SCEP was able to run its own update?? After starting the WU service, CPU utilization seems to maintaining normal levels.
I need to check some other servers.
-
The one server I have been troubleshooting on apparently updated its Definitions earlier this morning. Weird thing is that I disabled Windows Update service yesterday and it was still disabled when I just checked it. Maybe SCEP was able to run its own update?? After starting the WU service, CPU utilization seems to maintaining normal levels.
I need to check some other servers.
-
I manually updated definitions on few servers and it seems to have fixed the issue. There were two methods I tested:
- Download and install from latest full definition package. You can download from here: https://www.microsoft.com/en-us/wdsi/definitions
- From command line (Program Files\Microsoft Security Client), run MpCmdRun.exe -SignatureUpdate
- Proposed as answer by RyanConsol Tuesday, May 15, 2018 7:50 PM
-
Same here with updating the definitions manually. What I did was stop the Updates service (IE: net stop wuauserv). Run the manual definition file (from the link that BrandonM used), wait 30 seconds, then restart the Updates service. The servers seemed to jump back to 100% CPU for a few seconds and then was fine.
-
-
We run the mpam-feX64.exe file, CPU was back to normal during update checks and server was connecting to Wsus again (last report date).
Next day Wsus has new definition update files. Guess what, Problem returns. Anyone had the same experience?
-
I narrowed it down to our Wsus. Old definition updates did not retire automatically (still looking into that).
It looks like the troubled server 'hangs' on an old update or something. I retired all definition updates in Wsus except the new ones. Next i stopped the Windows Update service on the server, CPU load drops. Than execute commando : "C:\Program Files\Microsoft Security Client\MpCmdRun.exe" -SignatureUpdate. Windows Update service starts again, will go to 100% CPU, but drops after few seconds.
Wait until endpoint is finished with updating. (check >C:\Windows\Temp\MpCmdRun.txt)
From this point Windows Updates works fine, but i'm still a little worried about the next Wsus definition update files tomorrow.
-
Same here, problem returns when new definitions are available and the detection cycle runs. The poster below mentioned clearing this up on his local WSUS server, but our servers are just going through Microsoft Update for their AV definitions so I don't currently have control of what updates are available/approved.
It looks like Microsoft needs to look into this and officially address it via a patch to the Windows Update client, SCEP client, change to Microsoft Update, etc. to give us a long term fix. This OS isn't end of support yet...
Also, on a side note, I applied a policy to have one of the SCEP clients use the Microsoft Protection Center ONLY for AV definition updates and now the SCEP client's definitions update fine... but Windows Update service still hangs up. It looks like even if you tell SCEP not to use Microsoft Update for AV definitions that doesn't remove the production from the Windows Update search criteria when looking for updates.
-
This issue was caused by a large no of definition updates that were still unexpired on the Microsoft Update servers and evaluating the supersedence between them is what caused the high CPU issue for 10s of minutes. This has been fixed now by expiring the older superseded Definition Updates, as you can see here: https://www.catalog.update.microsoft.com/Search.aspx?q=KB2461484
Hope this helps,
Andrei
We could change the world, if God would give us the source code.
- Proposed as answer by Andrei StoicaMicrosoft employee Friday, May 18, 2018 7:35 AM
-
It appears Microsoft has fixed it.
We have about 20 non-R2 servers, they are all OK now.- Proposed as answer by Shera Singh Friday, May 18, 2018 7:11 AM
- Edited by rowgerk Friday, May 18, 2018 2:10 PM
-