none
Hyper-V cross forest Live migration

    Question

  • Hi everyone.

    I have a task to migrate Virtual machines from one Hyper-V cluster(A) to another Hyper-V cluster(B) in another domain forest.

    I prefer to do it by  sharing-nothing live migration with resource based Kerberos constrained delegation.

    I have rise 2 AD forests with 2 Hyper-V Clusters in my lab. There are no any firewall enabled on hosts or ports blocked between networks A and B. Each infrastructure emulated on 1 physical host under Windows 2016 because it supports nested virtualization and gives opportunities to rise Hyper-V cluster undo 1 physical PC. Thus i have that:

    Both Clusters runs at Windows 2012R2.  I have set transitive cross domain forests trusts between domains A and B and Kerberos constarined delegation between cluster hosts setup like described here  with little fixeds. Given example doesn't work because Get-ADComputer cant resolve computer object from trusted AD forest, thus command should contain - searchbase  and -server key to refer to trusted DC.

    Get-ADComputer -Filter 'Name -eq "destHost"' -SearchBase "DC=A,DC=domain" -Server "a.domain" | Set-ADComputer -PrincipalsAllowedToDelegateToAccount (Get-ADComputer -Filter 'Name -eq "srvHost"' -SearchBase "DC=B,DC=domain" -Server "nt.lan")

    After that the SIDs of delegated hosts appears in the PrincipalsAllowedToDelegateToAccount properties in host objects.

    Also I set concrete IP addresses on Hyper-V manager / Live migration setting. I think better to do it, because once i have migrate virtual machines between two clusters in single domain and different networks - and moving fails somewhere over migration until i set concrete IPs for LM traffic. 

    After that I tried to perform LM through Hyper-V manger GUI. 

    Process begins, in the destination folder was created Planned Virtual machines folder with future VM configuration files within. 

    After migration fails with error 0x80090303

     

    In the Hyper-V branch logs got such events:

    Source Host: 

    Event ID:20414
    The Virtual Machine Management service initiated the offline migration of virtual machine  'vins0004-nt' to destination host 'VINS011' (VMID A70F0D0E-4183-4E5A-A34A-E74AFBE84BD8).
    Event ID:20302
    The Virtual Machine Management Service failed to authenticate the connection for a Virtual Machine migration at the source host: The specified target is unknown or unreachable (0x80090303).
    Event ID:21024
    Virtual machine migration operation for 'vins0004-nt' failed at migration source 'VINS0010'. (Virtual machine ID A70F0D0E-4183-4E5A-A34A-E74AFBE84BD8)

    Dest Host:

    Event ID: 20402

    The Virtual Machine Management Service failed to authenticate the connection for a Virtual Machine migration at the destination host: %%2147778644 (0x80048054).

    This tread  tells that LM in that case may be successful only through PS Move-VM comandlet.

    I also tried such as shown here but get strange result.

    [vins0010]: PS C:\Users\username\Documents>  Get-VM  | % { Move-VM -DestinationHost vins011.%dest.domain% -IncludeStorage -DestinationStoragePath "C:\ClusterStorage\Volume1\$($_.name)" -verbose }
    cmdlet Move-VM at command pipeline position 1
    Supply values for the following parameters:
    Name: vins0004
    VERBOSE: Move-VM will move the virtual machine "vins0004" to computer "Microsoft.Virtualization.Client.Management.Server".
    Move-VM : Virtual machine migration operation failed at migration source.
    Failed to create folder.
    Virtual machine migration operation for 'vins0004' failed at migration source 'VINS0010'. (Virtual machine ID A70F0D0E-4183-4E5A-A34A-E74AFBE84BD8)
    Migration did not succeed. Failed to create folder '\\VINS011\VINS0010.2159109741$\{f887c148-ea23-4ab4-b2a1-d9ed8813c5f5}\vins0004\Virtual Hard Disks': 'The network 
    path was not found.'('0x80070035').


    I cant understand why got so strange result path: \\VINS011\VINS0010.2159109741$\{f887c148-ea23-4ab4-b2a1-d9ed8813c5f5}\vins0004\Virtual Hard Disks

    What does it mean VINS0010.2159109741$  ?

    Does anyone migrate machines between domains and clusters at all?


    Wednesday, February 08, 2017 3:48 PM

All replies

  • No, I've never tried to LM between domains. I want to say that I saw something somewhere that said it wasn't supported, but I can't find that now.

    However, there are two things that come to mind:

    • You can't perform a Shared Nothing Live Migration of a clustered virtual machine, even within the same domain. You must delete the virtual machine resources from the cluster before you can migrate it to any computer outside of the cluster. It appears from the errors that you're getting that you're already past that problem, but just so it's said.
    • 2016 has a new security model that prevents Kerberos delegation from working for Hyper-V. You must delegate to any protocol, not just Kerberos. Based on hits for 0x80090303, this appears to be the most likely culprit.

    I'm not promising that making those changes will work for you, but it seems fairly certain that they'll stop you if not changed.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Wednesday, February 08, 2017 11:59 PM
  • No, I've never tried to LM between domains. I want to say that I saw something somewhere that said it wasn't supported, but I can't find that now.

    However, there are two things that come to mind:

    • You can't perform a Shared Nothing Live Migration of a clustered virtual machine, even within the same domain. You must delete the virtual machine resources from the cluster before you can migrate it to any computer outside of the cluster. It appears from the errors that you're getting that you're already past that problem, but just so it's said.
    • 2016 has a new security model that prevents Kerberos delegation from working for Hyper-V. You must delegate to any protocol, not just Kerberos. Based on hits for 0x80090303, this appears to be the most likely culprit.

    I'm not promising that making those changes will work for you, but it seems fairly certain that they'll stop you if not changed.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Thanks!


    1. Of course i removed VM from cluster resource first, I meant it by default thus doesn't describe.

    2. As you may see my test Htper-V cluters (A and B) are virtual machines under 2016 host, clusters runs under 2012R2 not 2016.

    Thursday, February 09, 2017 9:32 AM
  • Fair enough. You kind of have a wall of text going on there with much more information than is necessary to explain your problem, and left a lot to be implied. I think you could have gotten it all across in two or three sentences.

    Anyway, you are at the point where this is a cross-forest delegation issue, not a Hyper-V issue, and you should change the context of your search to match. Searches for "cross-forest Kerberos delegation" might help you get started. I see some older hits that indicate that it may not work cross-forest, but those don't seem to take into account changes in AD after 2008R2. I'd consider asking in the directory services forums.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent contributor, not an Altaro employee. I accept all responsibility for the content of my posts. You accept all responsibility for any actions that you take based on the content of my posts.

    Thursday, February 09, 2017 6:04 PM
  • Hi,
    Are there any updates on the issue?
    You could mark the reply as answer if it is helpful.
    Best Regards,
    Leo

    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, March 02, 2017 7:53 AM
    Moderator
  • Hi,

    I think you have to forget about Live Migration, change the scenario and go with Export-VM migration scenario. :(

    There can be some options for consideration.. For example if you have Datacenter edition and necessary Win 2016 licences, you can upgrade cluster1 from 2012R2 to 2016 and implement Cluster-to-Cluster storage replica. That is just an idea - never tried this.

    Radek

    Thursday, March 02, 2017 8:10 AM
  • Hi, no updates, still no solution.
    Monday, April 03, 2017 2:58 PM