none
avhdx leftover from backup tool - how to merge or delete those files? RRS feed

  • Question

  • I've dome some tests with a 3rd party backup tool for Hyper-V. Unfortunately it does not work as expected. I removed it from the system. 

    Now I have a differential avhdx and corresponding *vhdx.mrt and *vhdx.rct files sitting around, for each virtual disk. 

    The avhdx is not visible in the hyper-v GUI as a snapshot (I did not create snapshots), and also Get-VMSnapshot will not show it at all (thus Remove-VMSnapshot won't work). I only see them in the filesystem.

    That would not bother me so much, but a side effect is that normal snapshots do seem to fail now.

    I tried to merge the avhdx into the main vhdx with "manage disk" in the hyper-v GUI, but the ID is not matching and I am not sure if and how to proceed. The GUI warns that there might be data loss. 

    How do I get rid of the .avhdx, .mrt and .rct files? As I mentioned, the usual way to merge snapshots does not seem to work. 

    Any pointer is very welcome. 

    Thanks

    Dan


    Saturday, November 30, 2019 9:29 PM

Answers

  • If you will go the import route, I would remove all traces of the existing VM and completely replace it with the exported version. That way, you can keep the unique IDs and there won't be any concerns with software trying to re-activate or anything like that.

    Other than its existence, I have no familiarity with the Nakivo product. I have seen behavior like this with multiple backup products, though. I think sometimes, thinks just get jammed up.

    Perhaps restarting the VMMS service or a host reboot will clear up whatever got hung.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    • Marked as answer by Dan_Hu_ Sunday, December 1, 2019 10:44 PM
    Saturday, November 30, 2019 10:48 PM

All replies

  • Just because an AVHDX exists does not mean that the VM has a checkpoint. If Get-VMCheckpoint returns nothing and the VM says it's running from VHDX, then the AVHDX is just leftover cruft. Delete it like any other file. Calling the backup API triggers the creation of the .MRT and .RCT files. You don't need to delete them. If you want to, you have to shut the VM down first.

    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Saturday, November 30, 2019 9:53 PM
  • Just because an AVHDX exists does not mean that the VM has a checkpoint. If Get-VMCheckpoint returns nothing and the VM says it's running from VHDX, then the AVHDX is just leftover cruft. Delete it like any other file. Calling the backup API triggers the creation of the .MRT and .RCT files. You don't need to delete them. If you want to, you have to shut the VM down first.

    Hello Eric,

    Thanks for your reply. 

    I did try to delete that DISK_GUID.AVHDX file, but then the VM would not start anymore. I noticed that the disk setting pointed actually to that DISK_GUID.AVHDX (in your words probably: "running from the AVHDX") and tried to change it to the original DISK.VHDX file. This resulted in another error where the Hyper-V-Manager told me that a merge is pending and therefore the disk cannot be changed. 

    I also tried the hint here: http://itproctology.blogspot.com/2008/06/how-to-manually-merge-hyper-v-snapshots.html. Actually, Hyper-V-Manager would not complain when wanting to merge .VHDX and .VHDX, but the result was that nothing happened. No change in size or even change in timestamp.

    So I changed back the DISK_GUID.VHDX to DISK_GUID.AVHDX, checked if that disk was still linked to DISK.VHDX and started the VM. Which worked fine.

    But now I'm back from where I started. I still have all disks in the configuration of both VM's point to the AVHDX files. 

    Do you eventually have another hint for me? I would appreciate it. 

    Thanks for the explanation on the creation of the .MRT and the .RCT files. Now I understand that part.

    Dan


    • Edited by Dan_Hu_ Saturday, November 30, 2019 10:25 PM spelling
    Saturday, November 30, 2019 10:24 PM
  • A pending merge means that it's trying to put it all back together for you and any tinkering is going to make it worse.

    I would export the running VM to a safe external location. That will give you a flattened version of the VM. Then shut it down to see if it manages to complete the merge on its own. If it doesn't, wipe out the VM and import.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Saturday, November 30, 2019 10:28 PM
  • Ah, import.. Yes, that is something I did not yet try. Anything I have to consider if I want to end up with an identical machine? 

    I will wait until tomorrow to check if the merge actually happens automagically. I am not so confident, though. That backup software (Nakivo) finished the initial backup about 2 hours before I tried to run it again to test the incremental backup capabilities. Since then another 8 hours have passed. Though with me tinkering...

    There was no mentioning that after a backup, I have to wait a period of time until I am allowed to do another backup. I've never ever had that kind of issues with Veeam. 

    Saturday, November 30, 2019 10:41 PM
  • If you will go the import route, I would remove all traces of the existing VM and completely replace it with the exported version. That way, you can keep the unique IDs and there won't be any concerns with software trying to re-activate or anything like that.

    Other than its existence, I have no familiarity with the Nakivo product. I have seen behavior like this with multiple backup products, though. I think sometimes, thinks just get jammed up.

    Perhaps restarting the VMMS service or a host reboot will clear up whatever got hung.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    • Marked as answer by Dan_Hu_ Sunday, December 1, 2019 10:44 PM
    Saturday, November 30, 2019 10:48 PM
  • Just a short one.

    Is it really an export I have to do? Just copy away the VHDX and then import that is not working?

    Saturday, November 30, 2019 11:34 PM
  • Export flattens the VHDX structure. Any problems with AVHDX parentage do not survive into the export. You don't have to do it, but you know that it will result in a good VM. Maybe it will merge on its own after a reboot or a VM shut down. Maybe it won't. Maybe trying to force it to merge will ruin the whole thing. Maybe it won't. Export/import is annoying and tedious, but it's a safe bet.

    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Saturday, November 30, 2019 11:38 PM
  • Hello Eric,

    Just to let you know, the "cruft" AVHDX were cleaned up after I rebooted the Hyper-V host. So your last idea to reboot was on point.

    I learned a lot during last night, also with your help. Many thanks for this. 

    One conclusion is that the backup tool I was testing has either a flaw, or I have not configured it right. It cannot be that using a backup tool, blocks the possibility to do checkpoints and the only (?) fix is to reboot the host. I did open a support case and will figure out what is wrong.

    Beside that, can you recommend Altaro for small (just 2 VM's) environments? 

    Dan

    Sunday, December 1, 2019 12:02 PM
  • I have seen this behavior with more than one backup tool. I cannot say for certain if the vendors are doing something wrong, if an API has undesirable behavior, or if things just go sideways sometimes. If any one tool does it every time, then I would avoid that tool. One tool doing it one time... I don't know that it means anything. As for Altaro, I have had a positive relationship with them for several years, so you should not trust me to give impartial advice on any backup product. Try a few out, see who has the most palatable usability/outcome/feature/support/price balance, and use what you like best.

    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Sunday, December 1, 2019 5:23 PM