none
CSV ran out of space during snapshot merge - now have too many AVHD files and a merge-pending state

    Question

  • While doing upgrade work on a VM running on a Windows Server 2016 cluster, multiple snapshots were created to provide a rollback if the upgrades had issues (they did, multiple times, so this was very helpful to have).  Once upgrade was completed and working correctly, I deleted the newest snapshot.  Another tech had, in the meantime, created a new VM that used most of the remaining space on the CSV.  So, my VM moved to Paused-Critical state when the merge used up the remaining space.  I immediately storage migrated other, still running VMs from the out-of-space CSV to another one.  I then resumed the VM that was in the Paused-Critical state.

    The VM appears to be fine and operating correctly.  I can't find any signs of data loss.  The snapshot I deleted no longer shows in Hyper-V manager.  Inspecting the disks show an intact differencing disk chain that has 3 links in it (Base VHD, AVHD for the remaining snapshot, and AVHD for "Now").  This is what I would expect.

    My concern is that there are 2 additional AVHD files on the CSV that are not chained into running configuration.  And an examination of the VM XML config file (version 2) shows that disk_merge_pending is true.

    My suspicion is that there when I shut down the VM, it will try to finish the merge that was aborted when space ran out on the CSV.  I am not sure what affect this will have on the state of the data, since and inspection of the current chain no longer includes these AVHD files.  Or, it may be that everything completed once space became available, and the flag simply didn't get cleared or the AVHD files deleted.

    Can anyone offer any insight into what I can expect when I shut this machine down?  Should my data remain intact?

    Obviously I will take a backup immediately before shutting down, but I don't want to have to do a restore if I don't have to.

    Thanks in advance for any help or insights.

    Tim Reid

    Monday, June 10, 2019 4:54 PM

All replies

  • Hi ,

    >> Inspecting the disks show an intact differencing disk chain that has 3 links in it (Base VHD, AVHD for the remaining snapshot, and AVHD for "Now").

    How did you check the disks show an intact differencing disk? Did you mean that click the Inspect button in the VM setting and then check inspect parent?

    >>My suspicion is that there when I shut down the VM, it will try to finish the merge that was aborted when space ran out on the CSV. 

    Please check the timing of 2 additional AVHD files to see if they align with your operation time. If yes, my suspicion is the same as yours.

    >>Can anyone offer any insight into what I can expect when I shut this machine down?  Should my data remain intact?

    It is hard to judge if the chain of virtual hard disks is corrupted. In order to protect your data, you might try to do a restore.

    Best Regards,

    Candy



    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   


    Tuesday, June 11, 2019 8:39 AM
  • Hi, Candy -

    How did you check the disks show an intact differencing disk? Did you mean that click the Inspect button in the VM setting and then check inspect parent? 

    Yes, I used inspect to determine the parent of each AVHD, starting with the one shown as the active disk, and following the parents all the way to the original VHD.

    Please check the timing of 2 additional AVHD files to see if they align with your operation time. If yes, my suspicion is the same as yours.

    The two AVHDs show timestamps that fall during the time period I was working on the server, and approximately correspond with when I made snapshots.

    It is hard to judge if the chain of virtual hard disks is corrupted. In order to protect your data, you might try to do a restore.

    That's basically my plan now.

    Thanks.

    -Tim



    Wednesday, June 12, 2019 2:11 AM
  • Hi Tim,

    You could mark the useful reply as answer if you want to end this thread up.

    Please feel free to let us know if you need further assistance.

    Thanks for your understanding.

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Wednesday, June 12, 2019 2:21 AM
  • Hi Tim, 

    Just want to confirm the current situations. 

    Please feel free to let us know if you need further assistance.                   

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Thursday, June 13, 2019 3:57 AM