none
Patch tuesday 08-13-2019 windows update - KILLED our 2008R2 PDC ( will not boot!) RRS feed

  • Question

  • Okay... after the most recent windows update, our DELL TS140 primary domain controller WILL NOT BOOT.

    Up till 2:30 in the morning yesterday patching workstations and servers. Everything was updating fine until I updated this one. Needless to say, this was an unwelcome surprise !

    Our other two 2008R2 machines have no problem ( they are Hyper-V VMs and not domain controllers, though) .

    Last night I restored our PDC from Sunday's Veeam backup, and it was fine again. UNFORTUNATELY, overnight it decided to AUTO UPDATE, and this morning I came in and it was DEAD AGAIN.

    Have restored AGAIN, this time disabling windows update and BITS services 

    Anybody got news on a FIX for this?? Um... pretty urgent.

    Wednesday, August 14, 2019 4:12 PM

All replies

  • Not sure why it was set to full auto update - I had previously set that server up to 'download but not install'... Anyway, I am wondering if the boot fail problem is related to THIS:

    https://support.microsoft.com/en-ca/help/4512816/devices-that-start-up-using-preboot-execution-environment-pxe-images-f

    perhaps KB4474419 must be installed BEFORE the rollup ??? ( looks like it wants to come in the other order... )




    • Edited by rotech8 Wednesday, August 14, 2019 6:05 PM
    Wednesday, August 14, 2019 6:00 PM
  • OK! Looks like I am on to something... I WAS able to reboot after installing JUST KB4474419...

    Next... the rollup [cue dramatic orchestral music]...

    Wednesday, August 14, 2019 7:40 PM
  • Any possibility that you use a Symantec AV product on this server? 

    https://support.symantec.com/us/en/article.tech255857.html

    https://www.zdnet.com/article/symantec-cannot-handle-sha-2-and-breaks-windows-7-and-server-2008-r2/

    Wednesday, August 14, 2019 7:47 PM
  • Nope - No Symantec - only use the native  MS-defender as AV there.

    still downloading  KB4512506 ....

    Wednesday, August 14, 2019 8:02 PM
  • Just confirmed : KB4512506 IS THE KILLAH! Fails the reboot, after applying that patch.

    This after the KB4474419 was successfully applied, and successfully rebooted.

    Interestingly (or not) as it was "applying", it got to 30% and then appeared to abruptly jump to the reboot stage (and subsequently failed).

    ROLLING BACK to the previous backup... disabling the updates till they get this fixed


    • Edited by rotech8 Wednesday, August 14, 2019 8:18 PM
    Wednesday, August 14, 2019 8:13 PM
  • Hi,

    Please check if the latest servicing stack update installed. If not, install it to improve the reliability of the update process to mitigate potential issues while installing the Rollup and applying Microsoft security fixes. 

    Servicing stack update for Windows 7 SP1 and Windows Server 2008 R2 SP1: March 12, 2019

    If the issue still occurs, it's recommended to keep KB4512506 uninstalled. We need to wait for the next Monthly Rollup update and check the symptom.

    Best regards,

    Yilia 


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Thursday, August 15, 2019 6:00 AM
    Moderator
  • Yes, the latest servicing stack is already installed. I have marked KB4512506  as hidden for now.

    That problematic patch has cost me NINE HOURS. I would have not wasted an extra nine hours of troubleshooting except for the fact it is marked "security".... 

    Thursday, August 15, 2019 3:07 PM
  • We just found a remote 2008 R2 virtualization host that failed to boot today. We are still trying to fix it with diskpart, chkdsk, etc, but i don't know if we will be able to succeed. The data seems to be there, but Bootrec /rebuildbcd says that it doesn't finds the OS :(
    Thursday, August 15, 2019 3:40 PM
  • This was exactly the same symptom our (physical) Domain Controller had. What the heck is the patch doing messing with boot records, I wonder?

    Also tried everything. The only fix was to restore from Sunday's Image Backup ( Thanks Veeam!!)


    • Edited by rotech8 Thursday, August 15, 2019 5:29 PM
    Thursday, August 15, 2019 5:28 PM
  • Hi,

    We have recevied some same response with this update issue, and will continue to confirm. It's suggested to keep this update uninstalled now. 

    If there is any updates, I will post here asap. 

    Thanks for your understanding.

    Best regards,

    Yilia  


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Friday, August 16, 2019 1:54 AM
    Moderator
  • Just to add, the KB4512506 is applied to one of my customers Test domains yesterday.

    VMWare Guest, PDC, McAfee - still running without errors.

    Just to see if it can be narrowed down to something specific.

    So two physical servers affected, one virtual server not affected. - Could be physical servers only.


    Friday, August 16, 2019 6:35 AM
  • I had a VM kept in the pre-login "applying updates" stage for hours. Tried to send a remote shutdown command. No luck. Had to cross my fingers and "unplug" the VM, then power on again, and then it booted in normal time. It has a GPT disk, but NOT the boot disk, just a data disk.

    Just for information i have 2008 R2 in that VM. It's a DC. And it's running ESET for file servers.

    The server that died was physical. Had Hyper-V role. The disk where the whole OS an VMs are is GPT. It also does have ESET running (all our computers do). It has UEFI.

    No workstations died until now. The update was installed on 10 of them.


    • Edited by frapetti Friday, August 16, 2019 1:15 PM
    Friday, August 16, 2019 1:13 PM
  • Here's a bit of trivia... ( perhaps irrelevant!)

    I just noticed that the only 2008R2 server that got killed by last week's security patches (the one that just happens to be our #1 domain controller) appears to have an EXTRA bit of DOT-NET framework installed ( mysteriously!)

    Looks to be a Chinese language-pack (??)

    hmmmm...

    I'm not sure how this got installed, but its not present on any of our other 2008R2 servers...

    Tuesday, August 20, 2019 6:46 PM
  • Hi,

    It is confirmed as a known issue of Aug updates. If you skipped install the 2019 7B  updates prior to installing 2019 8B updates OR fresh installed Win7 / W2K8 R2 then directly installed 2019 8B, the issue could be caused.

    The current resolution is to uninstall those updates and keep them uninstalled until the next action plan.

    If there is any update on this issue, I will post here ASAP.

    Thanks for your understanding.

    Best regards,

    Yilia 


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Wednesday, August 21, 2019 2:27 AM
    Moderator
  • Just to be clear, which KB numbers should we avoid to install until this issue is solved? I understand that it is KB4512506 , and nothing else? Is only Win 7 and 2008 R2 affected?


    • Edited by frapetti Thursday, August 22, 2019 1:49 PM
    Thursday, August 22, 2019 1:39 PM
  • Geeeee... LAST MONTH WASN'T HELLISH ENOUGH... it's gonna be a grand failure THIS MONTH AS WELL??

    Off to a great start here, gang!!

    Tuesday, September 10, 2019 8:54 PM
  • There is currently much gnashing of teeth going on here... ( see https://social.technet.microsoft.com/Forums/windowsserver/en-US/20134a24-f858-408f-9951-6248e32d52d0/patch-tuesday-09102019-windows-update-killed-our-2008r2-pdc-will-not-boot?forum=winservergen ) 
    Wednesday, September 11, 2019 4:15 AM
  • Is it trying to install the same patches again(Issue might not have been fixed yet) - or only new patches?

    And thanks for updating us, despite the level of frustration reaching critical.. :)

    Wednesday, September 11, 2019 6:06 AM
  • There were five patches total, and I didn't take note of what they were...

    Maybe If I look again it will give the list... OK, "5 important updates"

    KB4041083 (SQR dotnet)

    KB4514602 (SQR dotnet)

    KB4516065 (SMQR)

    KB4474419 (SU)

    KB3098786 (SU for dotnet)

    Wednesday, September 11, 2019 6:15 AM
  • Strange enough, this happened to two of my 2008 R2 servers on Monday 09/16/2019. The servers went right into the System Recovery screen. They are IBM System x3650 M4 servers. They use UEFI boot and the disk is GPT. I performed the following steps to fix it:


    Step 1: Add UEFI boot option in IBM Setup

    Add boot option in uEFI as below:
    UEFI -> Boot Manager -> Add Boot Option -> UEFI Full Path Option -> NO VOLUME LABEL -> <EFI> -> <Boot> -> bootx64.efi
    Input the Description -> Windows -> Commit Changes
    Change “Windows” from boot order to top, Save and reboot.

    source: https://forums.lenovo.com/t5/System-x-X6-M5-M4/x3650-M4-boot-failed/td-p/3971029

    Step 2: Boot Windows normally, the System Recovery screen will appear

    Choose "load drivers" if Windows doesn't see the disk. I used the ServerGuide 10 DVD, and searched the DVD for the "ibm_dd_sraid_2008R2" folder, that includes the drivers for my ServeRAID M5110e board.
    Then i choosed "Use recovery tools that can help fix problems starting Windows" (*), and then "Startup Repair".

    In my case this automatically fixed the problems on both of the affected servers, but only after performing both steps.

    NOTE: i understand that the recommended method is to "Restore your computer using a System Image", but i had no image created. Never needed one before, but no doubt i will look into them now.

    Or read it on spiceworks: https://community.spiceworks.com/how_to/164887-after-windows-server-2008-r2-update-the-server-won-t-boot-how-did-i-fixed-it

    I hope this helps.
    Regards.


    Tuesday, September 17, 2019 8:52 PM