none
Server 2016 VM Hangs shutting down/restarting when the Pagefile is on a different VHDX.

    Question

  • Howdy,

    We are working on creating some VMs running 2016 for testing.  Our normal setup for our 2012 R2 machines is a C drive (OS), D Drive (Data), and P Drive (Pagefile).  This has never caused us any issues.  When I set this same thing up on our Server 2016 VMs we run into problems where the VM wont' shut down or restart once we move the Pagefile to P.

    To do so I set C & D to No Pagefile and P to System Managed.  The first reboot usually works OK but after that when we try to restart or shutdown we just get to a plain blue background with the words "Shutting down" or "Restarting" and it never moved.  I tried restarting one last Thursday and was off Friday.  I checked it this morning and it still said Restarting...

    There is no crash or error or anything and I just have to power it off through the Hyper-V itself.  The VMs always boot back up OK and Event Viewer doesn't show anything except all the various things entered the Stopped State just before the hang.

    As soon as I put the Pagefile back on C all the problems fo away and I can shut down and restart all I want with no issues.

    Anyone run into this by change or have any ideas what the problem might be?

    Monday, November 21, 2016 6:12 PM

All replies

  • Are the C: and P: vhdx files on the same or different physical disks?

    With a properly sized VM, there should be no worry about trying to place the page file for improved performance.  The operating system still needs a page file to operate smoothly, but if you have memory sized correctly, you should be able to get by with a relatively small (4GB) page file for nearly any system.  And it will hardly add any IOPS to the environment.

    Page files are a bit of a hang over from the days of really expensive memory on small memory systems.  So people tended to make use of the page file to offset the cost of memory.  Today, memory is a very inexpensive way to increase the capacity/performance of many applications.  As a result, very rarely do I ever have a page file greater than 4GB, and even then, it is almost not used.


    . : | : . : | : . tim

    Monday, November 21, 2016 7:50 PM
  • We use a flash storage array so who knows where the actual file are but the C and P drives are in the same folder (CSV) separate from the D drive.  We use to have C, D< and P all in separated volumes but now we have C and P together which has worked fine with 2012 R2.  We generally set the P drive to be 20 gigs in size and just let Windows manage it itself.
    Monday, November 21, 2016 8:16 PM
  • I know it's not addressing your question about why you started having issues, but I would re-evaluate the reason for having a separate page file drive. My personal opinion is that it is completely unneeded in almost every case and just creates another thing to manage.

    I have never built a VM with a separate page file drive, so I'll have to build a clustered environment to try to test out what you are seeing.


    . : | : . : | : . tim

    Monday, November 21, 2016 8:53 PM
  • Morning all,

    I'm seeing this issue as well. Exactly the same but I use the D:\ as the page file to mirror how Azure does it so that when we want to move to/from and or have VMs created natively in Azure it makes understanding the drive letter locations and such a little easier.

    Also as for the main reason for having the page file on a separate VHDX (@Tim), personally I'm doing this for DR replication so I can exclude the pagefile disk from the replication model and save on replication traffic/bandwidth. I do exactly the same thing for the SQL database TempDB files.

    But yes - can we get a response from MS on this issue please? It is easily reproducible.

    Kind Regards

    Joe.

    Friday, December 02, 2016 8:08 AM
  • Seeing this exact same issue on my VMs with separate drives for the pagefile. To tim (and others with a similar mindset): Telling us to just switch it to C:\ is not an answer. There are very legitimate reasons for putting the pagefile off the system drive. Please don't assume out only reason is performance, In our case, it is to reduce unnecessary churn of data during Hyper-V replication. This has worked fine in other virtualized Windows operating systems. Only 2016 has this issue and Microsoft needs to fix it.

    Edit: FYI, this is not fixed as of the November update.

    • Edited by OTD Razor Tuesday, December 06, 2016 10:54 PM
    Tuesday, December 06, 2016 10:53 PM
  • "Telling us to just switch it to C:\ is not an answer"

    I did not tell anyone to 'just switch it to C:'.  I explicitly stated my response was not addressing the question, and then suggested re-evaluating the reason why it was split in the first place.  For many years I have seen two primary reasons for moving the pagefile off C:.  One was due to the small size of disks.  That reason has pretty much disappeared.  The second was for perceived performance reasons.  Again, that reason has pretty much disappeared as the usage of pagefiles has diminished significantly over the years.  However, even though the justification presented for moving the pagefile has diminished, I still see many people who have not re-evaluated their processes due to changes in technology.  Hence the suggestion to re-evaluate.

    You have re-evaluated and present a cogent reason with replication.


    . : | : . : | : . tim

    Wednesday, December 07, 2016 2:08 PM
  • I am unable to replicate what you guys are seeing. Server 2016 Guest OS with a separate VHDX for the page file.

    Same CSV, different CSV, dynamic VHDX, fixed VHDX, system managed PF, defined size PF; they all work fine and the VM shuts down/reboot just fine.

    I would suggest you open a support case with Microsoft.

    Wednesday, December 07, 2016 3:20 PM
  • We are experiencing the exact same issue.

    We disabled the page file on C and then create a system managed page file on D which is a separate virtual hard disk.  We are doing this to mimic the way Azure creates the D temp drive.  With this setup, if I wait at least 4 minutes of up time and then reboot, the VM hangs on restarting.

    I have installed the latest Dec Cumulative patch but this didn't fix the issue.  Something else interesting is that you must wait at least 4 minutes after the machine comes online before you attempt to reboot for the issue to appear.  If you reboot in less than 4 minutes the VM will reboot with no issues.  I also attempted to partition the primary VHDX so that D exists on the same drive but the issue was still there.

    Something else that I found out that if I don't disable the page file on C and leave both C and D to be System Managed, there is no issues with rebooting.

    Tuesday, January 03, 2017 6:03 PM
  • So I figured out that if I change the Write Debugging Information from Automatic memory dump to None, the reboot issues went away.  I am able to reboot with no issues having no pagefile on C and a System Managed Page File on D.  Seems like a bug to me.
    Tuesday, January 03, 2017 6:36 PM
  • Even though this seems like a bug, this following article clearly states that you must have a page file on the boot volume for automatic memory dump to work (this is on by default).

    So if you don't have a page file on your boot volume, turn this off.  This did fix my reboot issues:

    https://support.microsoft.com/en-us/kb/307973

    •To take advantage of the dump file feature, your paging file must be on the boot volume. If you have moved the paging file to another volume, you must move it back to the boot volume before you use this feature.

    • Proposed as answer by Paul Zirkzee Thursday, March 16, 2017 9:43 PM
    Tuesday, January 03, 2017 6:51 PM
  • Hi Robert,

    Thanks for sharing the information.

    We could wait for later official documents to see if there would be any related information.

    Best Regards,

    Leo


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Wednesday, January 04, 2017 1:13 AM
    Moderator
  • Discovered this bug today. In my deployments we always exclude the pagefile from DPM backups as described in the best practice of putting the pagefile onto another VHDX.

    Under Server 2012 R2 "Write Debugging Information" is set to Automatic so I can't see how this isn't a bug in 2016.

    Saturday, March 11, 2017 8:37 AM
  • I had exactly the same problem. Setting "System Failure -> Write debugging information" to "None" solved the problem.
    • Proposed as answer by gb51022 Wednesday, November 01, 2017 5:22 AM
    Thursday, March 16, 2017 9:45 PM
  • Even though this seems like a bug, this following article clearly states that you must have a page file on the boot volume for automatic memory dump to work (this is on by default).

    So if you don't have a page file on your boot volume, turn this off.  This did fix my reboot issues:

    https://support.microsoft.com/en-us/kb/307973

    •To take advantage of the dump file feature, your paging file must be on the boot volume. If you have moved the paging file to another volume, you must move it back to the boot volume before you use this feature.

    The article you linked seems outdated. Even though not for Windows Server 2016, the following article indicates that the page file does not need to be on the partition containing the installed OS. The partition containing the installed OS is often called the boot volume.

    https://support.microsoft.com/en-us/help/969028/how-to-generate-a-kernel-or-a-complete-memory-dump-file-in-windows-server-2008-and-windows-server-2008-r2

    In Windows 7 and Windows Server 2008 R2, to get a Memory Dump, the paging file does not have to be on the same partition as the partition on which the operating system is installed. To put a paging file on another partition, it is not mandatory to use DedicatedDumpFile registry entry.

    In my tests, I found that hangs on shutdown did not occur after changing "Write debugging information" to "Complete memory dump", "Kernel memory dump", "Small memory dump (256 KB)", and "(none)".

    In my tests, I found that changing "Write debugging information" from "Complete memory dump" to "Automatic memory dump" resulted in the first shutdown not hanging. However, the second shutdown did hang.

    In my tests, I found that changing "Write debugging information" from "Complete memory dump" to "Active memory dump" resulted in shutdown 1, 2, and 3 not hanging. After the 3rd shutdown, I forced the VM to demand more memory than was allowed by "Maximum RAM" then started shutdown 4. Shutdown 4 hanged.

    Therefore, my tests could be somewhat flawed because I usually did not do more than 1 shutdown after a change or force the VM to demand more memory than was allowed by "Maximum RAM".

    EDIT: After further testing, I found I could cause a hang on shutdown with every setting of "Write debugging information" except "(none)".

    • Edited by KSebion Thursday, May 11, 2017 7:34 PM New information
    Thursday, May 11, 2017 6:38 PM
  • After a long research (> 2h). There is no event, nowhere, and only affect windows server 2016, not 2008 R2, 2012, 2012 R2. The host is Windows server 2012 R2.

    Thanks to Robert Auten !!

    As other, we migrate SWAP to another drive for DPM. 

    After change to none, shutdown/restart works as usual.

    Tuesday, May 16, 2017 3:50 PM
  • So what we have here is a default configuration from Microsoft out of the box that prevents a machine from properly rebooting, and it's still the same way.

    I have 2 exchange servers that were using the default settings for the pagefile "system managed".  The default setting for System Failure is Automatic memory dump.

    The default settings have no pagefile on C:.

    The VMs do not reboot.

    If I built a car that didn't shut off someone would sue me.  If I left it that way, seemingly on purpose, people would hate me and shop somewhere else.

    Microsoft seems to be too busy doing other things like building Azure, failing to enter the mobile market, stunting game development with UWP and turning (poor) products into (poor) services for the sake of a buck.

    In 20 years I've never felt so discouraged at the direction of Microsoft.

    Would you please stop trying to copy the business model of Google and Apple and stick to writing the code that built the company?


    -=Chris


    • Edited by Progent.CT Wednesday, December 06, 2017 5:45 PM
    Wednesday, December 06, 2017 5:45 PM
  • Hi,

    the shutdown/restart problem is everywhere!

    Please Microsoft fix the error. It does not occur in Windows 2012 R2 (or earlier) either.

    In our cases, only the disabling of debugging helped.


    Danke und liebe Grüße Oliver Richter

    Wednesday, January 03, 2018 3:01 PM
  • The issue seems to be very common with Server 2016.  Sometimes the first reboot, maybe even a couple reboots is good, then you get stuck with a server that will NEVER reboot, it just is stuck "restarting".  Separating the pagefile into a separate VHDX is extremely common since it helps greatly with Hyper-V replication since there is no good reason to replicate the pagefile content.

    Please, Microsoft, fix the issue.  I've spent lots of time trying to just reboot after apply an update, but the VM would get stuck "restarting" forever.  And, if I power it off, no further updates can be applied since a "restart is required" registry entry is still in place since a clean restart will never ever happen unless the default dumping of memory to the pagefile is set to None.

    Sunday, January 07, 2018 2:21 AM
  • You have the opportunity to raise this directly to Microsoft product managers.  Forums are not designed to accomplish this.  If you want to report a bug or recommend a new feature directly to Microsoft product managers, use the Microsoft Uservoice web site - https://windowsserver.uservoice.com/forums/295047-general-feedback.  This method provides a way for others for vote on your recommendation, allowing the weight of numbers to help the product managers see what people really want to see accomplished.

    tim

    Monday, January 08, 2018 2:44 PM
  • Hi Tim,

    I did it!

    https://windowsserver.uservoice.com/forums/295047-general-feedback/suggestions/32877376-pagefile

    Hope the best!


    Danke und liebe Grüße Oliver Richter

    Wednesday, January 10, 2018 7:46 AM
  • Wow, I have struggled with this for a few months now.   Thank you so much everyone for uncovering this.  I also separate the page file sometimes so replication doesn't consume useless bandwidth.  Server 2016 Hyper-V refuses to shutdown if the page file is not on the C drive.  Pretty crazy bug

    I stumbled on other issues when diagnosing why some VM's would not shutdown or start up after automatic updates.  Random VM's hang on shutdown and or refuse to start back up when one or more of the following exists: formatted NTFS disk block size on volumes that host VHDX files that is not set to 64k, dynamic memory is enabled, NUMA spanning off.  Eventually, even manually killing the hung VM's they will not start or they stay hung shutting down or stopping.  The physical server must be rebooted to recover.

    2012 R2 never had any of these issues.

    Sunday, January 21, 2018 6:04 PM
  • @gettnmorebetter

    You describe exactly the same effects as us (all).

    And YES W2012 R2 never had any of These issues.

    Please vote for fixing this problem on: https://windowsserver.uservoice.com/forums/295047-general-feedback/suggestions/32877376-Pagefile


    Danke und liebe Grüße Oliver Richter

    Monday, January 22, 2018 8:20 AM
  • This is still happening.  I have my page file on a different VHDx

    Having a separate VHD removes the paging data from hyper-v replication plus backup (I run Dedup DPM) so the less trashy data I send the better.

    Even with properly sized memory and a pagefile located on P drive set to system managed, I do see noticeable improvements in the VM's performance, I suspect because the IO has been shifted to another drive.

    This bug needs to be resolved ASAP!

    Friday, February 02, 2018 11:54 AM
  • Figured it out....   With a stock system managed page file, if the size ever exceeds the free space available on c the os moves it to the next available drive.   At this point the VM refuses to reboot.   If you expand c so its larger than the page file- it reboots.     You can uncheck automatically manage page file size, re-enable it on c and disable it on d to get it back as it was at fresh install.  Reboots fine.  If you check the automatic manage box again after that the system will remove it from c again and create it on the next available drive.  It still reboots though.

    Im guessing the reference to the dump file is never changed in this process.  For 2008 and up, a DedicatedDumpFile registry key can be created to reference a new location.

    https://support.microsoft.com/en-us/help/969028/how-to-generate-a-kernel-or-a-complete-memory-dump-file-in-windows-ser

    Hope that helps

    Thursday, March 08, 2018 5:07 AM
  • Hi Blake Florin,

    I don't understand your remarks!

    In all our cases, the free space on the volume C is much much larger than the pagefile.

    e.g.:
    VM with 16 GB RAM
    Volume C - 100 GB - free space 65 GB  - no Pagefile - for System only
    Volume D - 500 GB - free space 150 GB - no Pagefile - for Data only
    Volume E  - 20 GB - free space 4 GB - 16 GB Pagefile (dedicated) - for pagefile only

    System will not reboot and not shutdown correctly. Only when we set "save debuginformation: None" the System shutdown and reboot correctly.

    How can your informations help us?


    Danke und liebe Grüße Oliver Richter

    Thursday, March 08, 2018 3:01 PM
  • Hi Blake Florin,

    I don't understand your remarks!

    In all our cases, the free space on the volume C is much much larger than the pagefile.

    e.g.:
    VM with 16 GB RAM
    Volume C - 100 GB - free space 65 GB  - no Pagefile - for System only
    Volume D - 500 GB - free space 150 GB - no Pagefile - for Data only
    Volume E  - 20 GB - free space 4 GB - 16 GB Pagefile (dedicated) - for pagefile only

    System will not reboot and not shutdown correctly. Only when we set "save debuginformation: None" the System shutdown and reboot correctly.

    How can your informations help us?


    Danke und liebe Grüße Oliver Richter

    Easy test...fire up a copy of that server and double the size of c and e, reboot.
    Monday, March 12, 2018 3:25 AM
  • Hi @Blake Florin and thanks but -

    What a crazy shit! ;-) Can't be serious?

    Double the size of volumes (OK only the needed C and E), but for what? 

    before: Volume C - 100 GB - free space 65 GB
    after: Volume C - 200 GB  - free space 165 GB -> 165 GB free space for what? - the system have only 16 GB RAM.

    I think there is a conceptual flaw by Microsoft.

    But good to know what (one) the solution is.


    Danke und liebe Grüße Oliver Richter

    Monday, March 12, 2018 9:05 AM
  • I was able to resolve this issue by changing the page file size to "Custom size" on the secondary drive and then setting the initial and max size as a number larger than the VM's memory. As an example, my VM had 27648 MB of Memory so I  set my page file initial and max to 32768. I had also increased the size of the C drive vhdx prior to changing the page file to custom, but shrunk it back down afterward and the reboot problems were still resolved.
    • Edited by mbiver Wednesday, March 28, 2018 10:02 PM
    Wednesday, March 28, 2018 9:40 PM