Warning: Copy a large amount of data

    General discussion

  • During complicated installation processes, it might be temporarly interesting, to take snapshots at important milestone points. But it is important to remove the snapshots when the configuration is terminated. Normally such tasks do not give any problems. But there are some exceptions in this kind of scenarios. Imagine you have fixed vhdx disks, that means that usually the diskspace consumption will not grow. But if you hadn't much diskspace left after creating the fixed virtual disks, you need to be very carefull. If you think to copy a large amount of data to a VM where you might have taken an initial snapshot (only during configuration time) it is important to know, that all your data won't be copied to the fixed vhdx disk, but rather to a avhdx file. So the data will be copied and your diskspace will decrease. When the physical disk is full, while the copy-process is still running, the VM will be saved. In such a scenario the VM will only be saved at the point, when you have approximately 30 MB of free space left on your phisical disk. The only way to go ahead is to immediately shut down your VM and to delete all snapshots. Depending on the amount of data which was already been copied, the main consuption of your important workingtime might just begin now: The merging-process is not just a simple copy-process. Even if you have only copied for example 150 GB of data (including large and little files), the merging time may take about 6 hours. If you wheren't able to delete any unused and large files on this disk before deleting the snapshots, the merge process runs at the low free physical disk space of about 30 MB.
    This situation can become reality just if you have forgotten to delete the snapshots prior to copy large amounts of data to a virtual disk. For this reason Microsoft should think about a mechanism within Hyper-V to avoid such a troubleshouting process. It is a fact, that everyone can take snapshots with just one click, without knowing anything of how the command works.

    Friday, July 19, 2013 7:07 AM

All replies

  • Nice one!

    Well, that deserves to be on the blog, rather then on hte forum.

    Friday, July 19, 2013 10:00 AM
  • Yes, this is exactly how it works.  (I don't consider this abusive, just venting)

    This is how it has worked since the original release of Hyper-V.  This is how it worked with VMware when they used to use COW (Copy on Write) files for snapshots (long ago).

    This is the nature of differencing disks.

    I would not consider it a warning, I would consider it something to be aware of.  This is a practice and procedure issue.

    When using snapshots a differencing disk is created.  This differencing disk 1) has a maximum size equal to its parent and 2) can overwrite all the data of its parent.  In theory, the differencing disk of a 40GB VHD can be filled up to 40GB - thus consuming 80GB of physical disk.  You merge and you are back to 40GB consumed.  The differencing disk is a block map that overlays the original.

    that is covered here:

    Consider this:

    In your model you use Fixed disks (I can only assume you are expecting some performance increase over dynamic) but, you also assume snapshots, which use differencing disks (dynamic by nature).  so, if you expected a performance gain over dynamic it is lost.

    Moving on, why not use dynamic disks by practice.  Then you only consume what is really data.  When you move to production, then convert to fixed if you feel that is important.  But remember that fixed with snapshots gives no win, only fixed without snapshots.  And as you use newer releases of Hyper-V the performance of dynamic virtual disk is on par with fixed virtual disk.

    And yes, you should know that taking a snapshot consumes more space (this happens in all hypervisors, but in different ways).  No matter what happens under the hood.  And taking a snapshot of a running VM also saves the RAM, consuming yet more space, so if space is a concern, power off the VM first.

    Brian Ehlert
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

    Friday, July 19, 2013 2:59 PM