none
BSOD on Hyper-V Host after Upgrading to 1903 RRS feed

Answers

All replies

  • I can confirm that I have encountered another BSOD after installing 1903. These did not occur in 1809 and prior.  I have updated with another memory dump in the provided link.


    Any assistance would be greatly appreciated.

    Thursday, October 24, 2019 8:29 PM
  • I managed to find a 1809 image and have reverted.  I will hunker down with that until 1909 is available and pray that this issue is fixed.  I will update here accordingly.
    • Marked as answer by Mike-E_wins Saturday, October 26, 2019 1:16 PM
    • Unmarked as answer by Mike-E_wins Friday, December 27, 2019 8:48 AM
    Saturday, October 26, 2019 1:16 PM
  • 5 days now of intensive usage under 1809 with ZERO BSODs occurring... this is not a hardware issue.
    • Marked as answer by Mike-E_wins Thursday, October 31, 2019 8:13 AM
    • Unmarked as answer by Mike-E_wins Friday, December 27, 2019 8:48 AM
    Thursday, October 31, 2019 8:13 AM
  • unfortunately 1909 does not resolve the problem.
    Thursday, December 12, 2019 11:07 PM
  • Whoa seriously, @fun_jay?  You are getting daily BSODs with 1909?

    I am still on 1809 and am holding off on upgrading as long as humanly possible.

    I have a feedback item here:

    https://aka.ms/AA6ekem

    Please upvote it when you get the chance.


    • Edited by Mike-E_wins Friday, December 13, 2019 9:44 AM
    Friday, December 13, 2019 6:42 AM
  • Lulz... looks like Feedback Hub no longer allows upvoting on bugs. 🤷‍♂️
    Friday, December 13, 2019 3:00 PM
  • The reports of this issue still occurring on 1909 got me to install it.  I'm 3 days in now without any issues.  I am now posting online to brag about my success and tempt any fate in store for me. 😅
    Monday, December 23, 2019 10:07 AM
  • Looks like this was still occurring with 1909 but was fixed in the December Cumulative Update.  So, this may no longer impact 1903, either.  Not going to find out, however.  I've been running 1909 with (fortunately) that December Cumulative Update and have had zero problems (well, except for this one which is not nearly as critical/disruptive).

    So, the team over there is alive. 😁  Thank you and Happy Holidays!

    • Marked as answer by Mike-E_wins Friday, December 27, 2019 8:52 AM
    • Unmarked as answer by Mike-E_wins Saturday, December 28, 2019 9:42 AM
    Friday, December 27, 2019 8:52 AM
  • FOOLED YOU! Unfortunately this is still an issue for 1909!

    Woke up this morning to the following crash:

    The computer has rebooted from a bugcheck.  The bugcheck was: 0x000001ca (0x0000000020c0d909, 0x000005d48e1bd987, 0x000005d48e1ee79f, 0x000000000000000a). A dump was saved in: C:\WINDOWS\MEMORY.DMP. Report Id: 39e35c2e-f918-4489-a77d-1efc1fafdee2.

    DMPs are here:

    https://1drv.ms/u/s!AgSt7TEEAf8En7ls-GF55eMV6VvMBg?e=mTpQhW
    Saturday, December 28, 2019 9:42 AM
  • *bump*
    Sunday, December 29, 2019 9:54 AM
  • 122719-10531-01.dmp was from 
    Windows 10 Kernel Version 18362 , so it was NOT from 1909, but still 1903.
    Sunday, December 29, 2019 12:45 PM
  • Thank you for taking the time to look into this matter, EckiS... for further clarity on my end, I never upgraded to 1903... I installed 1809 and then upgraded directly to 1909 from there.  In fact, the only option available to me is 1909:

    Sunday, December 29, 2019 1:14 PM
  • your screenshot proves that you are not on 1909, or Windows update would not offer you to update.
    Sunday, December 29, 2019 1:57 PM
  • That is correct @EckiS.  I have since reverted back to 1809 since, as you could probably imagine, having my OS blue screen at any given moment is not exactly an ideal environment.
    Sunday, December 29, 2019 2:10 PM
  • "this is still an issue for 1909" but then you post a dump file from 1903.
    and now you tell me you are on 1809.

    sorry, you will have to find someone else to look into your problem
    Sunday, December 29, 2019 2:15 PM
  • Thank you EckiS I appreciate any continued assistance.

    The thought crosses me that if you can tell me the expected kernel number I can reinstall 1909 to show you for certain.  I would need to know how to force upgrade to the expected 1909 kernel before doing so, however, as I have already mentioned I do not want to stay on a possibly errant 1909 any more than I would be required.

    Sunday, December 29, 2019 2:27 PM
  • oops, seems I have to apologize!

    Just took a dump file on a 1909 system:
    it displays "18362.1.amd64fre.19h1_release.190318-1202"
    so same as on your system.
    while winver displays 18363.535

    Thank you Microsoft (not) for your confusing version numbering
    Sunday, December 29, 2019 2:59 PM
  • Hah!  No worries @EckiS... this is not exactly simple stuff we're dealing with here.  I am glad you found that out because I was about to dive back into 1909, take some further screenshots and dive back out. :P
    Sunday, December 29, 2019 3:39 PM
  • The call stack looks like this one (from Windows Server), and you have an AMD Ryzen, too:
    Intel to AMD Processor BugCheck

    You could update your mainboard bios, seems you are not on the latest version for your 
    X470 GAMING PRO MS-7B79
    Monday, December 30, 2019 6:35 AM
  • Thanks for the additional information, @EckiS.  It would seem that if this was a hardware issue it would also occur in 1809.  We of course did all updates and driver installs for 1903 without improvement.

    In 1903 the BSODs seemed to occur every 24-36 hours.  I managed to get 3 days once (followed by another 24 hours).

    1909 I went six full days without a BSOD.  This led me to believe the problem had been fixed.  Until the morning that I returned to my station and all VM states had been wiped clean along with the host.

    Of course, 1809 there are ZERO BSODs which is why I am now on it. 😁

    All that stated I will go ahead and upgrade to the latest BIOS to be sure.  Trying to be a good citizen here.  Will update here with results.

    Monday, December 30, 2019 9:42 AM
  • OK on 1909 now with the latest BIOS.  Getting it installed was its own ordeal.  The BIOS file must be at the root of the USB drive, but I had it in a folder.  Upon "returning" to the UFEI menu the BIOS got bricked and I had to Clear CMOS to get back control. 

    On top of that, the profile I had "saved" did nothing when re-loading it back, so I essentially lost all my settings and had to reapply them from memory. NOT A HAPPY START OF THE DAY HERE. 😅

    Anyways, I will post here with any updates.  As mentioned earlier, my first streak was a full 6 days but I do not know if that is the new "standard" of failure in 1909 or if I was just lucky. :p

    Monday, December 30, 2019 12:00 PM
  • Well that did not take long.  Went out to the gym and returned to a reset machine due to a BSOD.  Updated the folder here with the DMP files, as well as exporting System Information:

    https://1drv.ms/u/s!AgSt7TEEAf8En7ls-GF55eMV6VvMBg?e=mTpQhW

    Monday, December 30, 2019 3:28 PM
  • same bugcheck and same callstack.
    I never had a AMD system, so are these really current and matching drivers?
    Seems strange that the dates are mismatching:
    \SystemRoot\System32\drivers\amdgpio2.sys Mon Sep 30 05:56:01 2019
    \SystemRoot\System32\drivers\amdgpio3.sys Mon Mar 14 11:19:36 2016
    \SystemRoot\System32\drivers\AMDPCIDev.sys Thu Apr 12 08:14:57 2018
    Monday, December 30, 2019 3:53 PM
  • I  opened your last dump file in Windbg and used the command
    lm sm ft
    (= list modules, sort by module name, display file time)

    Monday, December 30, 2019 4:15 PM
  • Ah ok thank you for the context, @EckiS.  Weird that my last comment asking about the dates is not showing?  My intention at this point is to make enough noise to get attention there in the Hyper-V group.  This appears to be an AMD-related issue that was introduced in 1903.  The concern being that 1809 will be sunset at some point, leading to the choice of either an unsupported OS or one that BSODs randomly while unattended.
    Thursday, January 2, 2020 10:17 AM
  • *bumping for awareness*
    Friday, January 3, 2020 10:40 AM
  • I have got exactly the same issue. Two days after upgrading from 1809 to 1909 my system BSODed with 0x000001ca (SYNTHETIC_WATCHDOG_TIMEOUT) when I was away. Some research showed that there are more people who experienced this problem after upgrading from 1809 and the system being idle but with VMs running. (https://www.windowsphoneinfo.com/threads/windows-10-ver-1903-with-hyper-v-bsod-synthetic_watchdog_timeout.239019/ and others)

    My system is a AMD Ryzen 5 3600 on a MSI X570A-Pro with the latest BIOS (H.60). After the crash I installed the latest Chipset-Drivers and switched the power management to "high performance" (maybe the problem is related to sleep states of the processor or something similar).

    By now (after 4 days) I did not experience it again but you cannot be sure, which leaves a bad feeling. Therefore I would also be really interested in this issue being solved.
    Friday, January 3, 2020 11:29 AM
  • Thank you webbie for your feedback.  Can I get you to please add a comment to the official feedback link here?

    https://aka.ms/AA6ekem

    FWIW I managed to get to 7 full days without any BSODs on 1909.  I was so convinced the problem was solved that I even put a thank you out on reddit.

    The next morning I was met with a BSOD. 😆

    If you go a few weeks without BSODing I would be interested in trying out the new chipsets on 1909 and see if it works for me.  Please do update this thread with your results.

    Saturday, January 4, 2020 1:12 PM
  • Actually @webbie70, have you tried without the latest chipsets and putting the power profile to High Performance?  I believe you are 100% in believing this is related to sleep states and/or due to being inactive.  This has only occurred while I was away from the machine and never while active use.

    Obviously, as you allude to, putting the machine in high performance mode is a work around, but not a viable long-term fix.  That will put a dent in the power bill for starters. :)

    Saturday, January 4, 2020 1:21 PM
  • No, I am not sure at all that changing the power profile helps. It's just a guess. My power meter shows about 10 watts more when the machine is idle on the high performance profile. That's not ideal but I can live with it, if it saves me from a crashed machine.

    I will keep you updated and post here if it happens again.

    BTW: With the Chipset-Drivers comes a Ryzen optimized balanced power plan which addresses issues mainly for game performance. They are tweaking timers and thresholds for p-state changes:

    https://community.amd.com/community/gaming/blog/2017/04/06/amd-ryzen-community-update-3

    Sunday, January 5, 2020 4:07 PM
  • Ah that is interesting @webbie70.  I can state for certain that I installed those drivers and did indeed use the optimized power plan and I still got BSODs in 1903.

    If you can go another week without BSODing I will attempt to do the same.  I might actually try it sooner as I am desperate to try any remedy at this point.


    • Edited by Mike-E_wins Monday, January 6, 2020 9:38 AM
    Monday, January 6, 2020 9:31 AM
  • OK @webbie70 you've inspired me... I had zero luck with 1903 and AMD drivers, but perhaps they identified the problem on their end since then?

    I ended up installing two sets of drivers.

    First was MSI from their site, dated 10/22/19:

    https://www.msi.com/Motherboard/support/X470-GAMING-PRO#down-driver&Win10%2064

    Then I followed your link above and installed what appears to be the same drivers but dated 11/25/19, and this also includes the Ryzen Balanced Power Plan:

    https://www.amd.com/en/support/chipsets/amd-socket-am4/x470

    The BSOD plan at this point is:

    1. Try Ryzen Balanced Power Plan (current)
    2. Try High Performance Power Plan
    3. Uninstall MSI-based drivers, reinstall AMD-based drivers
    4. Uninstall AMD-based driver, install MSI-based drivers
    5. Revert back again to 1809 :P


    • Edited by Mike-E_wins Monday, January 6, 2020 10:28 AM
    Monday, January 6, 2020 10:28 AM
  • Boo.... BSOD on the first night.  Next.

    The BSOD plan at this point is:

    1. Try Ryzen Balanced Power Plan
    2. Try High Performance Power Plan (current)
    3. Uninstall MSI-based drivers, reinstall AMD-based drivers
    4. Uninstall AMD-based driver, install MSI-based drivers
    5. Revert back again to 1809 :P

    @EckiS I have provided the minidump for you here in the 2019-01-06 folder in case you want to geek out over the new drivers and see anything interesting. 😁  The MEMORY.DMP is also currently syncing and should be available in a while.

    Tuesday, January 7, 2020 7:45 AM
  • Bleh... also the other thing with 1903 (probably related?) are random freezes while using Windows 10 clients, usually while media is playing (sound or video).  My VM guests will "hitch" and usually a left mouse click will unlock it.  Sometimes I have to restore window out to host and left-click the desktop there to unfreeze my VM guest instance.

    Reported here, FWIW: https://aka.ms/AA6wvmd

    Sorrynotsorry, but 1903/1909 feels very broken as compared to 1809.  Well, I should say the host does.  All my VM guests are running 1909 and run really well.   It's the hosts that feel broken and that's really the important part.

    Tuesday, January 7, 2020 2:03 PM
  • Now I had the BSOD again. So the high performance profile does not help. :(

    Like last time it happened while I was on lunch break. 4 VMs were running. 2 x Windows 10, an old Windows 7 and the DockerDesktopVM.

    I really hope it will be fixed soon.

    Tuesday, January 7, 2020 2:22 PM
  • Whaop... looks like this is not AMD-related after all:

    https://twitter.com/kassiesa/status/1214622961510035456

    Tuesday, January 7, 2020 7:44 PM
  • Looks like I have survived 1 day on High Performance.  That is not saying much, however.

    @Webbie70 what video card do you use?  Is it GeForce?  That seems to be the only connection at this point, besides broken software. 😆

    Wednesday, January 8, 2020 7:02 AM
  • @Mike-EEE: Sorry, my video card is an old AMD Radeon HD 6670.

    Wednesday, January 8, 2020 8:50 AM
  • No worries @Webbie70  that is actually a good thing... as that means this is more of a Hyper-V thing than a driver/hardware thing.

    Although it would seem setting High Performance addresses this issue, I am not sure I can tolerate the hitching that is also experienced in 1909, reported here: 

    https://aka.ms/AA6wvmd (Windows Insider)
    https://aka.ms/AA6ys0x (Non-Windows Insider)

    Those that I have spoken with that have seen this on 1909 have restored back to 1809.  I may do the same.

    Thursday, January 9, 2020 6:27 AM
  • So FWIW... I pinged one of my friends who happened to have worked on the Hyper-V Manager UI back in the day.  He sent a mail to the Hyper-V team and got me in touch with them.  They *think* (not too reassuring, but hey) that it's due to a problem fixed in an upcoming internal release slated for next month.  So, we're in discussions about having me install it and try it out.

    So I'll either have a working 1909 now or become a unwitting host to a crytocurrency miner operation. 😆

    I'll update this thread with any results/news.

    Saturday, January 11, 2020 5:18 PM
  • Any word on that update or any other fix?  I have just been dealing with these BSOD for a very long time now and would love to find a fix.  I started to work on replacing Hyper-V on the machine with a different hypervisor, but didn't finish.  Would rather stay with Hyper-V.

    James - Right Size Solutions

    Tuesday, January 21, 2020 2:10 PM
  • Boo... I am subscribed to this thread but am not getting any notifications.

    No word on any fixes, yet.  I am not even 100% confident that the "fix" I discuss above is the same issue we're experiencing here.  There were a lot of "maybe" and "mights" type of verbiage in the email and I have not heard back from them since.

    The suggestions I have are to revert to 1809 and MAKE NOISE.

    Specifically, post comments here:

    https://aka.ms/AA6ekem (Windows Insider)
    https://aka.ms/AA6yhil (Non-Windows Insider)

    And please UPVOTE this reddit thread here. 

    https://www.reddit.com/r/HyperV/comments/djqnhk/encountering_bsods_on_windows_10_hyperv_host/

    It's amazing the amount of comments in that thread but it only has 1 vote -- MINE. 😅  Quite confounding to think about, truly.

    Finally, you can also pester this Twitter thread here. 

    https://twitter.com/MikeTripleE/status/1210860831212220418

    It's tagged with the author of the only blog resource that is online about this issue and has worked @ MSFT for 40 years.  Ironically, his blog is named "ask bob" but Bob doesn't answer much when you actually do ask him something. 😆  I am thinking if he gets enough pings he'll use his 40 years of experience and send the right email to the right person, however. ;)


    • Edited by Mike-E_wins Sunday, February 2, 2020 11:59 AM
    Sunday, February 2, 2020 9:25 AM
  • BTW @Webbie70 has that High Performance setting panned out for you?  Did it solve your BSODs long-term?
    Sunday, February 2, 2020 9:26 AM
  • As I stated on January 7, I still get the BSODs even with high performance profile on. It's every few weeks now. I mitigate the problem a little by putting the VMs in saved state when not using them.
    Wednesday, February 12, 2020 7:11 AM
  • Ah sorry about that @webbie70 I am not getting notifications on this thread and am having to refresh it manually to see updates along with the other one.  That message got lost in the shuffle and I simply missed it after posting my AMD link. 

    It's good to hear about the mitigation in case others are forced to use 1903+.  I myself am sticking to 1809 and praying. :p

    Wednesday, February 12, 2020 7:56 AM
  • In case anyone is lost on my previous outburst, 1809 is the only defense against this issue, and it is being phased out in May.

    There was a new patch released earlier this week.  Has anyone had any more success on 1909 with the latest patches?

    Friday, February 14, 2020 9:40 PM
  • 84 days until being forced on a broken update.  Get ready to BSOD party y'all.
    Tuesday, February 18, 2020 5:36 AM
  • Finally heard back from the Hyper-V team.  Looks like a "possible" fix is slated last week of March.  So, guess we'll see then? 🤷‍♂️

    My confidence is not very high, but my hopes are. :p


    • Edited by Mike-E_wins Saturday, February 29, 2020 10:40 AM
    Tuesday, February 18, 2020 8:12 PM
  • OK all!  I have heard back from the Hyper-V team and they are saying 1909 build 1131+ (see edit below) has the fix in it.  Please give it a try and see how you do. I am psyching myself up for the upgrade, myself!

    Will update here accordingly.

    EDIT: wrong version was given by Hyper-V team, which was for 1809.  The build for 1909 is 18363.752.  You must install all cumulative patches to attain this version.
    Friday, April 10, 2020 6:10 PM
  • It might be my ignorance but I can only find builds 1131+ with the 1809 version. E.g. March 17, 2020—KB4541331 (OS Build 17763.1131).
    But Windows 10 version 1809 doesn't have this SYNTHETIC_WATCHDOG_TIMEOUT problem.
    Version 1909 seems to have different build numbers. E.g. March 30, 2020—KB4554364 (OS Build 18363.753).
    What is the right version?
    Saturday, April 11, 2020 6:07 PM
  • Derp!  I forgot to update this thread.  Sorry @BertKassies I am managing about a half-dozen threads around this issue and it's tough wrangling them all.

    Indeed you are correct.  The version given to me by the Hyper-V team was for 1809 which is super concerning as this problem does not occur in 1809 as you stated.  In fact, 1809 is the only known remedy for it.

    For 1909, you have to ensure all cumulative patches are installed, including and especially the optional ones, and that will bring you to 18363.752.

    I am going on day 3 now.  I have not encountered a BSOD nor the media stuttering issue in guest images that I did before.

    However, that is not saying much as it would take up to 8 days for a BSOD to occur.  The media stuttering would actually occur before, and was a canary of symptoms, IMO.  If I go a few more days without it, I will begin to hold out hope.

    FWIW, here's the stutter that I was experiencing in more detail:

    https://aka.ms/AA6wvmd (Windows Insider)
    https://aka.ms/AA6ys0x (Non-Windows Insider)

    Crossing fingers here.

    Sunday, April 12, 2020 7:20 AM
  • Ah geeze, details.  You said 753, @BertKassies.  I see now that I actually do not have that installed. :|  I have KB4541335 installed, which appears to be March 24th.  I do not see how to install KB4554364 (it doesn't appear as an option when checking for updates) but will investigate further.

    The Hyper-V team did not respond to my further questions around versioning on Friday.  It was later in the day so they may reply tomorrow (Monday), but as I am trying to keep the nagging to a minimum it will most likely be up to us to sort out the mess here.  Sorry for any confusion!

    Sunday, April 12, 2020 7:33 AM
  • I am going on day 3 now.  I have not encountered a BSOD nor the media stuttering issue in guest images that I did before.

    That's good news so far. Thanks a lot Mike for you comprehensive reports on this problem. Let's hope this fix holds.
    Sunday, April 12, 2020 5:36 PM
  • I suppose it can be downloaded from Microsoft Update Catalog (my non verified account doesn't allow me to post URLs;) ) and be installed manually. But perhaps it is better to wait for the 2020 April cumulative update next week which probably will include it.
    Sunday, April 12, 2020 5:51 PM
  • Wow!  I did not know you could do that.  I did a simple search and found the following:

    https://www.windowscentral.com/how-download-and-install-windows-10-cumulative-updates-manually

    I'm sure if someone needed to do so they would have done the same, but in case that is a good starting point. :)

    Going on day 4 here without issue.  I'm actually more concerned about the media hitching than anything as I think that was the same thing only it occurred during running of the VM rather than when it was idle.  Actually, I say "media hitching" but it was really the entire guest VM.  The media made it the most easily noticeable where it would simply freeze and the only way you could "wake" it was to click the mouse button. Very weird, but noticeable and has yet to happen yet.

    Anyways, fingers crossed still... a few more days of this and I might actually start to feel optimistic. 😆

    Monday, April 13, 2020 4:24 AM
  • Looks like there's another CU released yesterday KB4549951 that brings your build up to 18363.778.  This is not optional and was downloaded on every machine that I have tried.  In fact I had several instances auto-install and reboot (that's how I found out about it).

    Here's hoping that I can reboot my host OK. :P

    Wednesday, April 15, 2020 6:12 AM
  • Seems like 18363.752+ is the answer to this 6-month nightmare.
    • Marked as answer by Mike-E_wins Thursday, April 23, 2020 4:02 AM
    Thursday, April 23, 2020 4:02 AM
  •   Just in time for the next version upgrade to 2004 (build 19041)!

    Bill

    Thursday, April 23, 2020 7:29 AM
  • Haha right Bill!  Every "upgrade" is a cautious tread and an ordeal full of fear until proven otherwise now. :P
    Friday, April 24, 2020 1:40 AM
  • Seems like 18363.752+ is the answer to this 6-month nightmare.

    That's good news, Mike. I will try that next week.

    Until now I have a bad experience with version 1809.1131+ (KB4549949, OS Build 17763.1158). My Windows 7 VM is crashing and rebooting every few hours with that version.

    So, now I'm back to Windows 10 version 1809 build 1098 at the host. That version is running my Windows 7 VM for 1.5 year now, 24/7 with 60% load at the host, and never a single crash.

    Saturday, April 25, 2020 6:06 PM
  • Seems that Windows 10 1909 with April 2020 updates finally solves my problem.
    Now running a few days already with build 18363.836 without BSOD.

    Thanks a lot, Mike

    Monday, June 1, 2020 5:30 PM