WSUS - Join order has been enforced because a local join hint is used
-
Mittwoch, 5. Dezember 2012 14:25
We have an offline WSUS server that refuses to "download" updates after a certain point. First time we tried it got to 41MB, second time - 1083 and third time 2648....THe error log looks okay until the point where it says "Warning w3wp.7 DBConnection.onReceivingInfoMessage The Join order has been enforced because a local join hint is used"
Any ideas we are tearing our hair out here. Offline WSUS is probably the worst "solution" Microsoft has ever created. It simply doesn't work.
We have 2008 R2 SP2 running WSUS SP2 version .256
Alle Antworten
-
Donnerstag, 6. Dezember 2012 01:51Moderator
We have an offline WSUS server that refuses to "download" updates after a certain point.
Well, my first note would be that an OFFLINE server cannot download updates at all, because it is... well... OFFLINE!
Any ideas we are tearing our hair out here.
Only because I'm so really very tired of discussing this scenario, I would be inclined to say: Search the forum for conversations on "disconnected" servers -- you'll probably find several dozen from 2012 alone.
But, even though I'm so inclined, I won't actually do that. The answer is at the end of my reply.
Offline WSUS is probably the worst "solution" Microsoft has ever created.
And it couldn't at all possibly be that you've implemented the process incorrectly and it's just a matter of "operator error"?
It simply doesn't work.
Actually, it Does Work, and Thousands of organizations are doing so, including the Hundreds discussed in this forum who got it working perfectly after they started doing it correctly.
The issue you are encountering, most likely, is that you are improperly transferring the FILES from the connected server to the disconnected server, and thus the disconnected server cannot find the files it expects to be there.
There are three scenarios which can produce this behavior (and, as noted, discussed almost weekly in this forum):
- The connected server has not actually downloaded all of the files that it needs when the file export is initiated on the connected server, thus they simply do not exist on the disconnected server.
- The export was done on the WSUSContent folder, rather than the WSUSContent\* SUBfolders, resulting in the corruption of the ACLs of the WSUSContent folder on the disconnected server when the files were imported.
- The files were imported after the wsusutil import command was executed.
Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.- Als Antwort markiert Clarence ZhangModerator Montag, 24. Dezember 2012 02:34
-
Freitag, 7. Dezember 2012 14:17
Thanks for the reply.
First, we are using another server to download the updates and transferring these updates over. It is certainly the case that our process is incorrect, however, from reading online articles on setting up an offline instance it appears we have done them correctly. Second, the updates begin to download but stop at one of three spots each time. ~2.1GB, ~1.9GB, and ~2.7GB
In regards to your offered advice:
The connected server has not actually downloaded all of the files that it needs when the file export is initiated on the connected server, thus they simply do not exist on the disconnected server.
We did wait for the connected server to download ALL of the files before we initiated the export so there should be no missing files.
The export was done on the WSUSContent folder, rather than the WSUSContent\* SUBfolders, resulting in the corruption of the ACLs of the WSUSContent folder on the disconnected server when the files were imported.
The export of the data files was done on the WSUSContent\* Subfolders -- we also double checked all the ACLs of the WSUSContent folder on the disconnected server after the files were successfully updated. They are correct
The files were imported after the wsusutil import command was executed
If I am reading this correctly, we did do the import command after transfering the files into their proper location. at \WsusContent
One of the only things I see in the logs is that "The join order has been enforced because a local join hint is used" and occasionally we will see "found no more jobs, going to sleep for bits notification" Though that error hasn't appeared again in some time.
After writing this, we found that if we restart the BITS service each time the update install (offline wsus) stalls it will restart. This has to be done several times over. Any ideas on that?
-
Montag, 10. Dezember 2012 19:26
Lawrence,
We have still been working with WSUS to try and get it working on our end and have started from scratch to ensure each and every step is being done correctly.
I am curious to hear from your perspective exactly how WSUS works, specifically the import process (memory utilization and how the update metadata is being imported) and offline processes. It seems a lot of people ask the same questions over and over which must be fun for you. I noticed in some of your posts you mention that the key is getting people to understand the offline process and WSUS but it seems there is a major disconnect with users and how WSUS actually works. Looking to see if your willing to explain it here which might serve as a spot to get indepth knowledge. More than Microsoft offers in their documentation which from the number of posts seems to be somewhat unhelpful.
Also, for our issue. I have read over the Microsoft guide and I can confirm that we have:
- Matched advanced options
- copied the files using xcopy, maintained ACLs, file size is the exact same on offline and online, the folder structure has been maintained for all folders under the content directory
- export of the data was done with a member of a local admin group
- Bearbeitet FrankieFish1 Montag, 10. Dezember 2012 20:57
-
Dienstag, 11. Dezember 2012 14:50Moderator
Yes... the machine is actually trying to download files from Microsoft.com, but it's a disconnected server that has no Internet access. :-)After writing this, we found that if we restart the BITS service each time the update install (offline wsus) stalls it will restart. This has to be done several times over. Any ideas on that?
Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds. -
Dienstag, 11. Dezember 2012 14:57Moderator
I'd be happy to explain the process .... actually, I think I'll write a blog post for PatchZone, and link it back here.
The short answer for your situation is that the disconnected server should not be generating BITS download queue requests. The fact that it is means one of two things:
- The WSUSService cannot "see" the files in the ~\WSUSContent folder tree, or
- The files are not physically present.
Now, you actually say something interesting above: "copied the files using xcopy, maintained ACLs". In fact, if you are copying using XCOPY or ROBOCOPY, you do NOT want to "maintain ACLs"; you want the ACLs on the disconnected server to be inherited from the existing ACL of the WSUSContent folder. So, if you are forcing the ACLs of the connected server onto the disconnected server, then you've essentially done the same thing as if you had copied the WSUSContent folder itself.
Check the ACLs of a random sampling of the SUBfolders on the disconnected server. Verify that there are [a] no unresolved SIDs, and [b] the WSUSAdministrators group of the disconnected server is present in the ACLs.
Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.- Bearbeitet Lawrence GarvinMVP, Moderator Mittwoch, 12. Dezember 2012 01:15 Added hyperlink for PatchZone
- Als Antwort markiert Clarence ZhangModerator Montag, 24. Dezember 2012 02:34
-
Dienstag, 11. Dezember 2012 16:16
Thanks Lawrence I am really looking forward to your explanation of the offline process and get a more in-depth picture of WSUS.
If I am understanding your advice correctly, the files are physically present on the offline server itself. They are located in E:\WSUS\WsusContent and take up the exact amount of space as the files located on the online server installation down to the byte. I believe WSUSService can see the files in the WSUSContent folder as they do begin to "download" on the front page of WSUS. This then comes with another question....if you say there should be no BITS download queue requests...does that mean we shouldn't even see anything under the "Download Status" of the offline server? Ours begins to count up once the import has finished and we mirror the settings but each time it stops at different intervals. I had been running BITSADMIN /RESET /ALLUSERS to check what files it was stopping on. If I restart the service deleting the qmgr0.dat & qmgr1.dat files it will restart the "download status" count but will halt again usually on the same set of requests I saw prior to restarting the service. It sounds like though that I shouldn't even worry about seeing the updates "downloading" that after the import the metadata should be "attached" to the updates? I think the way we are thinking about it is that running an import with the .cab file takes the metadata in the file and "attaches" it to the download located in the WSUSContent folder. I am not sure if this is correct thinking or not, but it would make us think we shouldn't see anything "downloading" at all on the offline server. IF this is true...I don't understand why it is "downloading" them then. We have followed the process as we have read from the guide but still seem stuck.
Also, I did double check the permissions and they were inherited I mixed up my thinking there a bit. There were no missing SIDs and WSUS Administrator group did exist. I randomly checked about 20 folders.
-
Mittwoch, 12. Dezember 2012 01:21Moderator
I believe WSUSService can see the files in the WSUSContent folder as they do begin to "download" on the front page of WSUS.
The fact that they "begin to download" is the first clue that all is not right.
This then comes with another question....if you say there should be no BITS download queue requests...does that mean we shouldn't even see anything under the "Download Status" of the offline server?
Yep. :-)
I had been running BITSADMIN /RESET /ALLUSERS to check what files it was stopping on.
Well.. by definition... it's going to stop on the FIRST file queued.. since it's a disconnected server! Yes?
It sounds like though that I shouldn't even worry about seeing the updates "downloading" that after the import the metadata should be "attached" to the updates?
Well, actually, to reconcile the imported data with the files present in the file system can take several hours. But that task should produce =0= requests for file downloads!
I think the way we are thinking about it is that running an import with the .cab file takes the metadata in the file and "attaches" it to the download located in the WSUSContent folder.
Correct.
...it would make us think we shouldn't see anything "downloading" at all on the offline server.
Correct.
I did double check the permissions and they were inherited I mixed up my thinking there a bit. There were no missing SIDs and WSUS Administrator group did exist.
OK. Good to know. The bad news: It would seem that we are now in uncharted territory. :-//
Now the question is this: Why does the WSUSService think the files are "missing"?
Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds. -
Mittwoch, 12. Dezember 2012 13:59
Lawrence,
Thanks for this information its really helping us understand WSUS a bit more. I agree I can't find those who have the same issues as us but yet have completed the steps correctly. Our offline server no matter what we do under "Download Status" shows updates downloading. It moves very quickly but stops suddenly. If we are not suppose to see this at all...then I honestly am at a loss as to the reason we are seeing these updates "download." We have followed the process again and again and yet we continue to see these messages. We have many different security settings and GPOs in this environment and have begun to try and find out of something else is at play, but at the moment we are just lost.
Unfortunately I am unable to give you any logs without manually typing them out which I can certainly type some lines out if need be. Is there anything we can provide you or do?
Thanks again
-
Freitag, 14. Dezember 2012 22:48Moderator
Our offline server no matter what we do under "Download Status" shows updates downloading. It moves very quickly but stops suddenly.
Okay, let's clarify something here. I did not mean to say that you might not see a status display on the console.. I said that WSUS should not issue download requests to BITS. I can see where my terse response might have confused this question.
Creating a download request is bona fide evidence that a file is either missing or has the wrong ACL. Think of the process in this way:
- The import process identifies approved updates that need files.
- A separate async process actually processes a SQL UPDATE to the update record, when the file has been downloaded, and marks it "Files are downloaded" and updates the Download Status on the main page of the console.
So while the import is off identifying missing files; this other task is matching up the files that exist with the updates that need files. If the process "stops suddenly", that's because a file is missing and now the process that matches up existing files with updates is waiting on a file to be downloaded from BITS -- which ain't never gonna happen on a disconnected server.
And herein lies the fault with assuming that because the number and size of files on one server matches the other, all is hunky dory. Consider the possibility that a needed file is physically missing from the connected server, which would also make it missing from the disconnected server.
Go back to the connected server, run wsusutil reset. See if it identifies any missing files. If so, after it downloads them, then redo your file transfer and metadata import.
Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.- Als Antwort markiert Clarence ZhangModerator Montag, 24. Dezember 2012 02:34

