Move Virtual Machine Storage on a Clustered vm = Disk Read Error?
-
Thursday, November 29, 2012 7:44 PM
Having an issue when moving Virtual Machine Storage on a Clustered vm is breaking (corrupting?) the vhdx.
Setup:
2-node Hyper-V Server 2012 (the free one) Cluster connected to an EqualLogic PS6100E using a CSV. (2) Teamed (Hyper-V) LAN nics and (3) iSCSI MPIO nics each.
Starting Location:
- VM named "vv-2012" is built from an ISO and running on a server in the node - NOT added to Failover Cluster Manager as a role yet.
- Disk lives at "C:\ClusterStorage\volume1\vv-2012\Virtual Hard Disks\vv-2012.vhdx"
- VM lives at ""C:\ClusterStorage\volume1\vv-2012\Virtual Machines\xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.xml"
- Machine is Running at Login Screen directly after setting the local Administrator password
- Running current Integration Services
Test 1 - From Starting Location - Move VM Storage using Hyper-V Manager - Move just the vhdx one folder up ("C:\ClusterStorage\volume1\vv-2012\vv-2012.vhdx")
- SUCCESS - vhdx moved and machine still running
Test 2 - From Starting Location - Import into Failover Cluster Manager - Move VM Storage using Failover Cluster Manager - Move just the vhdx one folder up ("C:\ClusterStorage\volume1\vv-2012\vv-2012.vhdx")
- FAILED - Disk Read error in the running VM as soon as the move starts (see below image)
Test 3 - From Starting Location - Import into Failover Cluster Manager - SHUT DOWN RUNNING VM - Move VM Storage using Failover Cluster Manager - Move just the vhdx one folder up ("C:\ClusterStorage\volume1\vv-2012\vv-2012.vhdx")
- SUCCESS - Started VM - no errors
Working on the, now, broken vhdx:
- Searching technet (and the Internet - Google/Bing) returns almost exclusively P2V conversions using disk2vhd.exe from Sysinternals. That is NOT the case here - reading these posts talks about partitions not being imported (boot partition) or incorrect IDE/SCSI settings on imported vhd's.
- Attaching the vhdx to another vm (or my local machine using Disk Management) and running chkdsk returns "Corrupt master file table. Windows will attempt to recover master file table from disk." "Windows cannot recover master file table. CHKDSK aborted." (also tried with /F even though screenshot does not show that - same result) - I see the 2 partitions (boot and system) but cannot browse them - marked as RAW
- Running TestDisk returns invalid MFT (I think, not an expert on this program).
Summary:
- Moving vm vhdx's that are turned off either through Hyper-V Manager or Failover Cluster Manager work fine
- Moving vm vhdx's that are running through Hyper-V Manager work fine
- Moving vm vhdx's that are running through Failover Cluster Manager breaks (corrupts?) the vhdx.
So my questions are 3 fold:
- Can anyone else reproduce this?
- Is it a Hyper-V / Cluster bug?
- Can I recover the vhdx?
Wasn't sure if I should post here or Cluster forum? Both?
:) -Andrew
Images:
All Replies
-
Monday, December 03, 2012 9:12 PM
Hi Andrew,
What version of the HIT Kit are you using? I believe that the 4.5 version of the EqualLogic HIT Kit that supports Windows Server 2012 and CSVs only released very recently (maybe a few days ago). And it is still early customer access also so you'll need to login to the http://support.equallogic.com website to grab the download.
I'm not an EQL technical expert but I heard something similar to this that was resolved by disabling the ODX feature in the registry. You may give that a whirl to try and avoid in the future if you cant get the HIT kit. I htink you can check out this link for some info on disabling ODX http://technet.microsoft.com/en-us/library/jj200627.aspx#DeployODX_Step4TestODXPerformance
-Brian
- Proposed As Answer by Andrew Teets (Concord) Thursday, December 13, 2012 12:26 AM
-
Wednesday, December 05, 2012 1:57 AMModerator
Hi,
How are things going? I just want to check if the information provided was helpful. If there is any update or concern, please feel free to let us know.
Best Regards,
AidenIf you have any feedback on our support, please click here
Aiden Cao
TechNet Community Support -
Thursday, December 06, 2012 12:46 AM
Hi Brian -
Excellent idea! I'l be testing this later this week!
We are on HIT 4.0.
- Andrew
-
Thursday, December 13, 2012 12:23 AM
Success!
I first tried just upgrading to HIT 4.5 (Early Release only right now) and no change.
Then I disabled ODX in the registry (Set-ItemProperty hklm:\system\currentcontrolset\control\filesystem -Name "FilterSupportedFeaturesMode" -Value 1) as described in Brian's post above.
Rebooted and retested - all tests are working!
Is this a problem with Windows Hyper-V then?
Thanks for all the help!
-Andrew
- Edited by Andrew Teets (Concord) Thursday, December 13, 2012 12:24 AM
- Edited by Andrew Teets (Concord) Thursday, December 13, 2012 12:24 AM
- Edited by Andrew Teets (Concord) Thursday, December 13, 2012 12:25 AM
- Edited by Andrew Teets (Concord) Thursday, December 13, 2012 12:27 AM
- Edited by Andrew Teets (Concord) Thursday, December 13, 2012 12:27 AM
-
Thursday, April 04, 2013 11:44 AM
Hi Andrew do you know if there has been any hotfix for this due that we have exactly the same issue vhdx and now with vhd
Andrew has the problem reoccurred since you had success with the registry change
-
Tuesday, April 30, 2013 4:53 PM
Well after a few months of not being able to move storage (given, not that big of a deal, but to MAKE SURE we didn't do it was a pain) I was pointed to KB2796995.
I fire up a test server and add it to the cluster (getting current Windows Updates in the process) and try to recreate the failure. CAN'T GET IT TO FAIL.
I try the exact steps listed above on the existing cluster hosts (that have had updates over the last several months) - CAN'T GET IT TO FAIL.
Looking through the updates applied since my original post I don't see anything DIRECTLY related to CSV & VHD(X)'s - but there are a lot of updates I'm not 100% certain what they did.
So, where I am now - either:
- A Hotfix was installed through Windows Update that fixed the issue.
- The issue exists and I just haven't hit the exact circumstances to make it reappear
The hotfix I linked to above is not installed on the machine (and thus not part of Windows Update as of today).
This is not reproducable with or without the ODX registry fix mentioned a few posts above.
I'm not sure if I should apply the hotfix or not.
If/when we need to do a storage migration we're going to do it on a test machine before the live one.
Side note - KB2813630 may also be of interest to anyone else reading this - it deals with CSV volumes going offline when backing up through Hyper-V.
Thanks
-Andrew
- Edited by Andrew Teets (Concord) Tuesday, April 30, 2013 4:53 PM
- A Hotfix was installed through Windows Update that fixed the issue.


