none
Hyper-V 2019 VM Cluster Replication RRS feed

  • Question

  • Ever since upgrading our 2 Hyper-V clusters to 2019 from 2012 R2 when trying to set up a new VM replication I get errors. Works for existing VMs replicating that were created before the upgrade but any new VMs fail. Enabling the replication seems to create the VM fine on the replica and creates drives at 4,096KB ready for replication but trying to start the initial replication fails with the error:

    Start-VMInitialReplication : Hyper-V failed to start replication for virtual machine 'VM': Operation aborted
    (0x80004004). (Virtual machine ID E3E5EDD5-66CA-48AB-8F52-CA63A3A6AEC5)
    Hyper-V could not find the virtual machine 'VM' on the Replica server and will connect to the Hyper-V Replica
    Broker in the next retry interval. (Virtual machine ID E3E5EDD5-66CA-48AB-8F52-CA63A3A6AEC5)
    Replication operation for virtual machine 'VM' failed. (Virtual machine ID
    E3E5EDD5-66CA-48AB-8F52-CA63A3A6AEC5) (Primary server: 'Host1', Replica server:
    'Broker')
    At line:1 char:1
    + Start-VMInitialReplication -VMName 'VM'
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (Microsoft.HyperV.PowerShell.VMTask:VMTask) [Start-VMInitialReplication],
       VirtualizationOperationFailedException
        + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StartVMInitialReplicationCommand


    Doing the same process through the failover cluster gui I get a similar error. Says the replication was enabled successfully however initial replication could not be started. And goes on to say Hyper-V could not find the virtual machine on the replica server.

    Has this yet to be recognized as a bug in Windows Server 2019?
    Tuesday, March 26, 2019 10:36 AM

Answers

  • I fought with this for weeks after our upgrade to 2019, and I think I finally found a solution.  I hope this works for other people.

    As everyone else has mentioned, I would enable replication on a VM and I would see the VM get created on the target cluster, but then the initial replica would fail and it would state that the VM couldn't be found.

    I tried dozens of things to get it working and what I finally found is that initialization would work so long as the target VM was in the root of the CSV.

    For example:

    By default replication wants to bury the VM at a path like c:\ClusterStorage\Volume1\Repl\BlahBlah

    Once the VM is created on the target cluster, move the VM to a folder in the root of the CSV like C:\ClusterStorage\Volume1\MyVM and then try the initialization again.  Worked for me every time since I started doing that.

    Friday, April 19, 2019 10:23 PM

All replies

  • Hi ,

    This is a quick note to let you know that I am currently performing research on this issue and will get back to you as soon as possible. I appreciate your patience.
    If you have any updates during this process, please feel free to let me know.

    Best Regards,

    Candy


    Please remember to mark the replies as an answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com   

    Wednesday, March 27, 2019 9:34 AM
  • This is a known issue. I have found many threads about this.  There is a long running one here:

    https://social.technet.microsoft.com/Forums/Lync/en-US/ecd99229-b9fe-45c7-b35e-207c4903f73a/hyperv-cluster-replica-broker-error-quotobject-could-not-be-foundquot-from?forum=winserverhyperv

    I was able to do a work around using Veeam's  community edition to preform my replication and failovers. 

    you can download it here: https://www.veeam.com/virtual-machine-backup-solution-free.html

    Wednesday, March 27, 2019 9:21 PM
  • Any news?
    Thursday, April 4, 2019 8:14 AM
  • Hi,

    To test, can you please choose a test VM, manually create a VM in the same name at the replica server and try to initiate the initial replication?

    Are there any anti-virus software? can you disable the same at both the ends and then try to initiate the initial replication?

    Regards,
    Bala N


    Thursday, April 4, 2019 10:01 AM
  • Tried this but no luck I'm afraid. When enabling the replication it already creates the replica on the other cluster but then says it can't find it. I believe this is a known bug as several people have cases open with Microsoft about it see here:

    https://social.technet.microsoft.com/Forums/Lync/en-US/ecd99229-b9fe-45c7-b35e-207c4903f73a/hyperv-cluster-replica-broker-error-quotobject-could-not-be-foundquot-from?forum=winserverhyperv

    Although I'd like to know if there is any kind of eta on a fix
    • Proposed as answer by John..Adams Friday, April 19, 2019 10:18 PM
    • Unproposed as answer by John..Adams Friday, April 19, 2019 10:18 PM
    Friday, April 5, 2019 7:07 AM
  • I fought with this for weeks after our upgrade to 2019, and I think I finally found a solution.  I hope this works for other people.

    As everyone else has mentioned, I would enable replication on a VM and I would see the VM get created on the target cluster, but then the initial replica would fail and it would state that the VM couldn't be found.

    I tried dozens of things to get it working and what I finally found is that initialization would work so long as the target VM was in the root of the CSV.

    For example:

    By default replication wants to bury the VM at a path like c:\ClusterStorage\Volume1\Repl\BlahBlah

    Once the VM is created on the target cluster, move the VM to a folder in the root of the CSV like C:\ClusterStorage\Volume1\MyVM and then try the initialization again.  Worked for me every time since I started doing that.

    Friday, April 19, 2019 10:23 PM
  • Thanks a lot!!!!!!     This worked for me.     I have been dealing with this for months.
    Monday, April 22, 2019 4:26 PM
  • Didn't work for me.

    Moved the target VMs to the root of CSV, the same error:

    32370 "Hyper-V could not find the virtual machine on the Hyper-V Replica Broker.".

    Hyper-V 2019 standalone replication to Hyper-V 2019 Failover Cluster, both Datacenter editions.

    Wednesday, May 8, 2019 8:20 PM
  • Hi Guys,

    I had the same problem, moving to the root of the CSV worked fine but now seeing a few other issues.

    1. Changing the Replica path in the Replication Broker, you can't change it to the CSV unless you and the replication Broker role are on that Node which is currently the owner of the CSV

    2. even after changing the replica path, replication still going to the previous path

    John

    Wednesday, May 15, 2019 8:19 AM
  • Excellent, that worked. Thanks for sharing!
    Wednesday, July 24, 2019 7:29 PM
  • Thanks!

    not that i enjoy having to deal with the extra step (most likely bug) but big thanks for the fix! 

    Works fine after you move the VM..to Root of CSV in PerVM Folder, Can be done during Replication enabling and after VM is created 

    Wednesday, August 28, 2019 8:05 PM
  • We were still having the problem with some of the VMs when moving the VM files into their own folder.  We wound up creating a new folder, but with a shorter name and then it worked. 🤷‍♂️  

    First folder name: webserver1, didn't work.

    New folder name: ws1, worked.

    We had other folders that were longer in length, so not sure if it was purely because we shortened the path, or just new folders with new names.  We were just happy to get the replication to work.

    Friday, September 20, 2019 2:54 PM
  • October and still no patch to fix this issue as promised????

    Tuesday, October 15, 2019 10:38 PM
  • We were still having the problem with some of the VMs when moving the VM files into their own folder.  We wound up creating a new folder, but with a shorter name and then it worked. 🤷‍♂️  

    First folder name: webserver1, didn't work.

    New folder name: ws1, worked.

    We had other folders that were longer in length, so not sure if it was purely because we shortened the path, or just new folders with new names.  We were just happy to get the replication to work.

    Thanks - worked in our Infrastructure too :)

    Im not sure but in Replica Broker I've deleted backslash in path and our working Path is atm like:

    C:\ClusterStorage\volume1\repl

    Default path ( when you click Browse) is:

    C:\ClusterStorage\volume1\repl\

    Monday, November 4, 2019 3:51 PM
  • 6th November, 2019 and so far no patch to fix the issue.
    Wednesday, November 6, 2019 9:19 AM
  • Also having the same issue. Can't believe MS has let this lapse for so long! Trying to do extended replication from 2016 to 2019 cluster.
    Thursday, November 14, 2019 6:35 PM
  • I was told by a Microsoft tech that the fix for this was going to be released late November 2019 but was then pushed back to 3rd week of January 2020.  The John Adams' workaround above resolved the issue for me.  I created individual CSVs for each VM with only 1 subdirectory at the root of the CSV for the VM to be placed in, moved the VM storage to that folder after the first failed initial sync.  After that, the initial syncs were successful.  Even then, I sometimes had to shorten the folder name.   
    • Edited by Mark McNeff Tuesday, November 19, 2019 2:21 PM
    Tuesday, November 19, 2019 2:07 PM
  • OK so I'm having same error regarding the VM not being found this is on 2016 cluster.. this might be abit diffrent though.

    I was able to initially replicate the VM to the backup site. and successfully run a tesfailover so i decided to do a planned fail-over with revers replication. however once the VM failed over and hyper-v tried to revers the replica i get the Error 32370 VM not found on the replica... after scratching my head for hours i released the original VM (now the replica as i failed it over) has the windows server 2016 ISO still attached to the DVD drive.. so i removed it and hit replicate... and what do you know it started working.

    not sure if this will help anyone else... but make sure those pesky ISO images have been removed. i had a similer issue when we used to stoer the ISO images in a non CSV storadge and live migrate would fail spectacularly so we started storing the ISO images on a CSV.. looks like ISO images also cause issues in replication... annoyingly.


    Yep... Need more coffee

    Monday, January 20, 2020 4:57 PM
  • Its is the 3rd week of January any word on this patch still having the issue on 2019 the work around is a PIA.
    Tuesday, January 21, 2020 5:03 PM
  • I think i have just found a much easier workaround to this. Move the replica broker to the same host that is the current owner of the CSV you are pointing it too.

    It will then allow you to select the top level folder of the CSV.

    In other words, the replica broker role and the CSV owner at the time, needs to be the same host..

    Thursday, January 30, 2020 9:35 AM
    • Proposed as answer by ionisit Monday, February 3, 2020 11:39 PM
    Monday, February 3, 2020 11:39 PM
  • fixed with this KB update : https://support.microsoft.com/en-us/help/4534321/windows-10-update-kb4534321
     I don't see anything in there relating Hyper-V, Replication? Am I missing something?
    Tuesday, February 4, 2020 9:18 PM
  • the patch i posted works, i've verified
    Tuesday, February 4, 2020 11:12 PM