none
SCVMM Standard CU2 2016 - Cluster Crash and unable to delete a host not belonging to the cluster

    Question

  • I have written before about this product.

    My setup was, 3 hosts in a cluster, and 4 standalone host all connected on the same subnet with one SCVMM2016 Standard CU2 to manage them all.

    Yesterday I deleted 3 standalone without any issue, the 4 fourth troughs me a lot of errors and when I look at the bucket error , I get the following group of errors.(the forum makes this smaller, not me)


    The fourth server produces a lot of errors, 

    Warning (25354)
    Cluster node Server35.fqdn(IT IS NOT PART OF THE CLUSTER, IT WAS ONLY ADDED BECAUSE THERE WAS ZERO OTHER OPTIONS TO MOVE THE VM FROM STANDALONE TO THE CLUSTER)

    Recommended Action
    Add the pending cluster node under VMM management(NO IT DOES NOT BELONG TO THE CLUSTER).

    Error (2927) (PLEASE ALLOW ME TO DELETE THIS SERVER)
    A Hardware Management error has occurred trying to contact server 'Server35.FQDN' :n:CannotProcessFilter :HRESULT 0x8033801a:No instance found with given property values. .

    WinRM: URL: [http://SERVER35:5985], Verb: [INVOKE], Method: [GetVersion], Resource: [http://schemas.microsoft.com/wbem/wsman/1/wmi/root/scvmm/AgentManagement]

    Unknown error (0x8033801a) (Please allow me to delete the host, please provide this option for me SCVMM2016, a force delete, just as you have in Active Directory when a Domain Host cannot uninstall AD correctly)

    Recommended Action
    Check that WinRM is installed and running on server 'SERVER35.FQDN'. For more information use the command "winrm helpmsg hresult" and http://support.microsoft.com/kb/2742275.

    WinRM is saying it is okay on SERVER35

    Error (20413)
    VMM encountered a critical exception and created an exception report at C:\ProgramData\VMMLogs\SCVMM.9d2f8e81-2d93-4738-b818-9af845ff80fe\report.txt.

    Recommended Action
    See the report for exception type, exception message, stack trace and other relevant details. Also check the Windows Event Log for additional troubleshooting information.


    Error report created 14-02-2017 09:38:40
    CLR is not terminating

    ----------------------------------------------------
    ------------ exceptionObject.ToString() ------------
    ----------------------------------------------------
    Microsoft.VirtualManager.DB.CarmineSqlException: Unable to connect to the VMM database because of a general database failure.
    Ensure that the SQL Server is running and configured correctly, then try the operation again. ---> System.Data.SqlClient.SqlException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_tbl_WLC_VHD_VHD". The conflict occurred
    The statement has been terminated.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
       --- End of inner exception stack trace ---
       at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlContext.ExecuteNonQuerySP(String spName, Action`1 parameterAdder, Action`2 afterExecutionAction)
       at Microsoft.VirtualManager.DB.VmmDbTransaction.Microsoft.VirtualManager.DB.IVmmDbTransaction.ExecuteSP(String spName, Action`1 parameterAdder)
       at Microsoft.VirtualManager.DB.ImageLibrary.VHDDBAccess.Remove(Guid vhdId, IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDB(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDBForHostRemove(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.<>c__DisplayClass9.<RunSubtask>b__4(IVmmDbTransaction _tx)
       at Microsoft.VirtualManager.DB.SqlContext.Transact(CallSite callSite, String connectionString, WindowsIdentity identity, Action`1 action)
       at Microsoft.VirtualManager.DB.TransactableDbContext.Microsoft.VirtualManager.DB.IVmmDbConnection.Transact(Action`1 action)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveHostTask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.DB.SqlContext.Connect(Action`1 action)
       at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)
    *** Carmine error was: DatabaseFailure (2605); SQLServer; 547
    [s#1a6cee] ExecuteNonQuery(this: (SqlRetryCommand#[cdf3c]) { cmd = (SqlCommand#7f8c54) System.Data.SqlClient.SqlCommand, errorTitle = (string) "errorTitle", sqlContext = (SqlContext#[1e910]) { [rw] RemoveGuestAgentsFromHostSubtask.RunSubtasktx = (SqlTransaction#32628bc) System.Data.SqlClient.SqlTransaction, currentTxRecord = (SqlTransactionRecord#[7ba9]), constructionIdentity = (WindowsIdentity#11dbe72) System.Security.Principal.WindowsIdentity commandList: { n=2 { tx:(SqlTransactionRecord#[7ba9]) Q:"dbo.prc_WLC_GetVHDById(VHDId=8b5d0f5d-e477-413e-a998-17a1a8ca2016)", @14-02-2017 09:38:40 }, { tx:(SqlTransactionRecord#[7ba9]) NQ:"dbo.prc_WLC_RemoveVHD(VHDId=8b5d0f5d-e477-413e-a998-17a1a8ca2016)", @14-02-2017 09:38:40 } } txRecordList: { n=1  { tx#31657 RemoveGuestAgentsFromHostSubtask.RunSubtask() +00:00:00.0047773 ReadCommitted commands: { n=2 { tx:(SqlTransactionRecord#[7ba9]) Q:"dbo.prc_WLC_GetVHDById(VHDId=8b5d0f5d-e477-413e-a998-17a1a8ca2016)", @14-02-2017 09:38:40 }, { tx:(SqlTransactionRecord#[7ba9]) NQ:"dbo.prc_WLC_RemoveVHD(VHDId=8b5d0f5d-e477-413e-a998-17a1a8ca2016)", @14-02-2017 09:38:40 } } } } , conn = (SqlConnection#3246232) { State: Open Database: VirtualManagerDB DataSource: SERVER99\HYPERV PacketSize: 8000 ServerVersion: 13.00.1722 StatisticsEnabled: False WorkstationId: Server99 } }, behavior = Default, failFast = (bool) False })
    [s#1a6ced] VHD.RemoveFromDB(this: (VHDX#[14e80]) { objLock = (CarmineObjectLock#[a78c]) { lockType = Delete, objectID = 8b5d0f5d-e477-413e-a998-17a1a8ca2016, objectType = VirtualHardDisk, taskID = 5d7b4b4d-d67b-4ee4-b64b-eca4a345553f }, data = (VHDData#[bb42c]) { id = 8b5d0f5d-e477-413e-a998-17a1a8ca2016, LastUpdatedTimestamp = 12-02-2017 22:18:36, objectType = VirtualHardDisk, ObjectState = 0, name = "Server37_5E525B89-9064-4A49-AA0F-34015693A5AC", Enabled = True, Accessibility = Internal, description = "auto-discovered", CreationTime = 16-01-2017 14:21:19, modifiedTime = 12-02-2017 22:18:36, GrantedToList = ∅, vmHostId = 76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, hostType = None, Location = "D:\HyberV_VM\Server37_5E525B89-9064-4A49-AA0F-34015693A5AC.avhdx", HostVolumeId = 9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96, objectSize = 1492123648, srcObjectId = 00000000-0000-0000-0000-000000000000, srcObjectType = VirtualHardDisk }, originalContent = (VHDData#[bb42d]) { id = 8b5d0f5d-e477-413e-a998-17a1a8ca2016, LastUpdatedTimestamp = 12-02-2017 22:18:36, objectType = VirtualHardDisk, ObjectState = 0, name = "Server37_5E525B89-9064-4A49-AA0F-34015693A5AC", Enabled = True, Accessibility = Internal, description = "auto-discovered", CreationTime = 16-01-2017 14:21:19, modifiedTime = 12-02-2017 22:18:36, GrantedToList = ∅, vmHostId = 76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, hostType = None, Location = "D:\HyberV_VM\Server37_5E525B89-9064-4A49-AA0F-34015693A5AC.avhdx", HostVolumeId = 9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96, objectSize = 1492123648, srcObjectId = 00000000-0000-0000-0000-000000000000, srcObjectType = VirtualHardDisk }, pendingChildAdditions = ∅, isNewObject = False, host = (Host#[14e6e]) { objLock = (CarmineObjectLock#[a783]) { lockType = Delete, objectID = 76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, objectType = VMHost, taskID = 5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, IsInTaskCache = (bool) True }, credential = (SecureCredential#[1f2c]) { user = "hypervraa", domain = "fqdn", password = "<password omitted from trace>", encrypted = False }, data = (HostData#[bb3b5]) { id = 76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, LastUpdatedTimestamp = 14-02-2017 08:38:02, Flags = AvailableForPlacement, HostGroupID = 9db34b98-02c3-44e6-8470-b807682ce203, AgentServerID = 09b94958-3298-47ea-831c-6d73bc4c81d9, OverrideHostGroupReserves = False, NumaSpanningEnabled = True, CPUPercentageReserve = 10, NetworkPercentageReserve = 0, DiskSpaceReserveMB = 10240, MaxDiskIOReservation = 10000, MemoryReserveMB = 2048, VhdPaths = (String[][0]#3a72623) { }, VMRCEnabled = True, VMRCPort = 2179, SecureVMRCEnabled = False, VMRCCertificateAvailable = False, VirtualizationPlatformScheme = "viridian://Server35.fqdn", SupportsLiveMigration = True, EnableLiveMigration = True, LiveMigrationMaximum = 2, LiveStorageMigrationMaximum = 2, UseAnyMigrationSubnet = False, MigrationSubnet = (String[][0]#1e884f0) { }, MigrationSubnetUserManaged = (Boolean[][0]#15cd791) { }, MigrationAuthProtocol = CredSSP, VirtualServerState = Running, VirtualServerVersion = (Version#603807c9) 6.3.9600.18377, VirtualServerVersionState = UpToDate, IsDMZHost = False, UseCACert = False, IsNonTrustedDomainHost = False, UserName = "", Password = "<password omitted>", LogicalProcessorCount = 24, PhysicalProcessorCount = 2, CoresPerProcessor = 12, L2CacheSize = 1536, L3CacheSize = 15360, BusSpeed = 100, ProcessorSpeed = 1995, ProcessorModel = "Xeon", ProcessorManufacturer = "Intel", ProcessorArchitecture = 9, ProcessorStepping = "179", TotalMemory = 68636917760, OperatingSystemName = "Microsoft Hyper-V Server 2012 R2 ", OperatingSystemVersion = (Version#60380fff) 6.3.9600, OpticalDrives = "E:;", IsoRWRepositoryAvailableBytes = 0, CpuUtilization = 6, AvailableMemory = 60168, Custom = (String[][9]#2997716) { (null), (null), (null), (null), (null), (null), (null), (null), (null) }, MaximumMemoryPerVM = 1048576, MinimumMemoryPerVM = 32, SuggestedMaximumMemoryPerVM = 512, IMAPIInstalled = False, AvailableMemoryTierredPerfCounterID = b9f6757c-0965-4834-86f7-7aca6f05e3a8, CpuTieredPerfCounterID = b9a3c28b-8ab6-43d9-948e-a1e22ba50454, ClusterNodeStatus = ClusterNodeStateUnknown, SecureMode = True, SslTcpPort = 5985, SslCertificateImportedByVMM = False, SshTcpPort = 0, IsRemoteFXRoleInstalled = False, IsCPUSLAT = True, TimeZone = (Int32?) 60 (0x3c), HostVolumeIDs = (List`1<Guid>#e1aa8e) { n: 2 962883e9-9aaf-49a7-9988-3308aa11a362, 9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96 }, HostDiskIDs = (List`1<Guid>#256e709) { n: 3 63b6a56b-01d4-4ce8-87b5-90a801a10e30, 4d9a5fbf-96d9-4be6-945a-ef3add3fef1d, 053bf4ba-be66-4b8d-8094-f64f66f63c07 }, StorageFileShareIDs = ∅, HostGPUIDs = ∅, InstalledVirtualSwitchExtensionIDs = (List`1<Guid>#2d5731) { n: 3[4] ae9ede92-9cbf-42ba-8d39-6a4a40ba144d, 254c28ae-6fe1-41b5-bc19-b777de44d3a4, 2510a9b6-b4b3-4058-93e9-d38c87ee0831 }, HostFibreChannelHbaIDs = ∅, HostSASHbaIDs = ∅, HostInternetSCSIHbaIDs = ∅, PhysicalMachineID = 4765543f-54be-416d-8073-4e59d7e26963, OobType = None, IsPowerOptimizationControlled = False, RunAsAccountID = 1daa42f2-6a8b-42af-bdc0-fb79d7b4c047, CustomPropertyIDs = ∅, HealthMonitorIDs = (List`1<Guid>#3123ae3) { n: 10 cf44fc85-300f-4612-9cb8-2e5f8ac1b151, 6f3a183f-f520-4126-ac10-32b0709ab60b, 81e771ec-7f54-42c0-ba58-384423f67fbc, 7eaf40f0-b33b-409b-bb14-3cc8e20d02f7, 0b3e14e4-cdb9-490b-b2b8-54b6958e0fb9, b2128b34-96ee-4a06-8bbf-766c02cb4b61, 8fcc841b-34be-4be6-87c8-856aef1ec040, fd93fc7e-aff7-4fc0-b0a6-8c903d4cf2bc, 3b855827-39ee-44c0-8def-9a3d24fac775, 2833240c-e469-45ec-a4f7-aae325c019e2 } }, server = (RemoteServer#305c24e) Microsoft.VirtualManager.DB.Adhc.RemoteServer, agentVersionState = (VersionState) { UpToDate } }, needToReleaseCloudLock = (bool) False, needToReleaseHostOrLibLock = (bool) False, differencingDiskParentChanged = (bool) False }, tx: (VmmDbTransaction#[a27c]) { startTime: (DateTime#4afe4a3d) 14-02-2017 09:38:40 sqlContext: (SqlContext#[1e910]) }) in s:\btvsts\3125\private\product\engine\ImgLibEngine\VHD.cs:line 1034
    [s#1a6cec] VHD.RemoveFromDBForHostRemove(this: (VHDX#[14e80]) {  }, tx: (VmmDbTransaction#[a27c]))
    [s#1a6ce1] RemoveGuestAgentsFromHostSubtask.RunSubtask(this: (RemoveGuestAgentsFromHostSubtask#[bb3a8]) { subtaskId = 639e6281-270a-4da2-a984-092498884560, description = (SubtaskDescription#102f00) Remove guest agents from virtual machine host, children = (NotifyList`1<ISubtask>#1de391f) Microsoft.VirtualManager.Utils.NotifyList`1[Microsoft.VirtualManager.Engine.ISubtask], taskContext = (TaskContext#5034a4) Microsoft.VirtualManager.Engine.TaskRepository.TaskContext, parentSubtask = (RemoveHostTask#[bb3a0]) { subtaskId = f390a89f-725e-4180-b085-194268555980, description = (SubtaskDescription#2b9a09d) Remove virtual machine host, children = (NotifyList`1<ISubtask>#27e44fd) Microsoft.VirtualManager.Utils.NotifyList`1[Microsoft.VirtualManager.Engine.ISubtask], taskContext = (TaskContext#5034a4) Microsoft.VirtualManager.Engine.TaskRepository.TaskContext, isInitialized = True, isTopLevelSubtask = True, shouldSkipCompletedSubtasksOnRestart = False, state = Running, clientTraceProvider = (TraceProvider#31a8031) Microsoft.VirtualManager.Trace.TraceProvider, authData = (TaskAuthorizationData#faa33a) Microsoft.VirtualManager.Engine.TaskAuthorizationData }, isInitialized = True, isTopLevelSubtask = False, shouldSkipCompletedSubtasksOnRestart = False, state = Running, clientTraceProvider = (TraceProvider#31a8031) Microsoft.VirtualManager.Trace.TraceProvider }, dbConnection: (TransactableDbContext#[236a1]) { constructionCallSite = (CallSite) { SqlContext.Connect }, sqlContext = (SqlContext#[1e8ff]) { [rw] SqlContext.ConnectconstructionIdentity = (WindowsIdentity#3823de5) System.Security.Principal.WindowsIdentity commandList: { n=93 { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, @ObjectType=VMHost, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_Host_Get(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HealthMonitor_GetAllIDsByAgent(@AgentServerID=09b94958-3298-47ea-831c-6d73bc4c81d9)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_GetIDsByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_BTBS_CustomPropertyValue_GetCPValIDByObjID(ObjectID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostDisk_GetIDsByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_StorageFileShareAssociation_GetShareIDsByServerIDAndType(@ServerID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, @ServerType=VMHost)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_GetHostFibreChannelHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_GetHostSASHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_GetHostInternetSCSIHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostGPU_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_NetMan_InstalledVirtualSwitchExtension_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_GetHostFibreChannelVirtualSANByHostId(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_RBS_GetRunAsAccountConsumer(RunAsAccountID=, ObjectID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_RBS_GetRunAsAccountById(RunAsAccountID=1daa42f2-6a8b-42af-bdc0-fb79d7b4c047)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_RBS_UserRoleSharedObjectRelation_GetSharedListForObject(ObjectID=1daa42f2-6a8b-42af-bdc0-fb79d7b4c047)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=962883e9-9aaf-49a7-9988-3308aa11a362, @ObjectType=VMHostVolume, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_Get(@VolumeID=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96, @ObjectType=VMHostVolume, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_Get(@VolumeID=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostDisk_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 [+00:00:00.0156496] }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=63b6a56b-01d4-4ce8-87b5-90a801a10e30, @ObjectType=VMHostDisk, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostDisk_Get(@DiskID=63b6a56b-01d4-4ce8-87b5-90a801a10e30)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=63b6a56b-01d4-4ce8-87b5-90a801a10e30)", @14-02-2017 09:38:34 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=4d9a5fbf-96d9-4be6-945a-ef3add3fef1d, @ObjectType=VMHostDisk, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostDisk_Get(@DiskID=4d9a5fbf-96d9-4be6-945a-ef3add3fef1d)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=4d9a5fbf-96d9-4be6-945a-ef3add3fef1d)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:34 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=053bf4ba-be66-4b8d-8094-f64f66f63c07, @ObjectType=VMHostDisk, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostDisk_Get(@DiskID=053bf4ba-be66-4b8d-8094-f64f66f63c07)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=053bf4ba-be66-4b8d-8094-f64f66f63c07)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_GetHostFibreChannelHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_WLC_GetDRProtectedVMIdsByHost(HostId=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_TR_Task_GetTaskWithSubtasks(@TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_TR_Task_GetUserRolesForTask(ObjectID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_AgentServer_Get(@AgentServerID=09b94958-3298-47ea-831c-6d73bc4c81d9)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_AgentServerRelation_GetByAgent(@AgentServerID=09b94958-3298-47ea-831c-6d73bc4c81d9)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_AgentServer_Get(@AgentServerID=09b94958-3298-47ea-831c-6d73bc4c81d9)", @14-02-2017 09:38:34 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=09b94958-3298-47ea-831c-6d73bc4c81d9, @ObjectType=AgentServer, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_ADHC_AgentServerRelation_GetByAgent(@AgentServerID=09b94958-3298-47ea-831c-6d73bc4c81d9)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_UM_GetAllUpdateServers()", @14-02-2017 09:38:34 }, { Q:"dbo.prc_TR_Task_GetTaskWithSubtasks(@TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_TR_Task_GetUserRolesForTask(ObjectID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:34 }, { Q:"dbo.prc_TR_Task_GetTaskWithSubtasks(@TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_TR_Task_GetUserRolesForTask(ObjectID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)", @14-02-2017 09:38:39 }, { NQ:"dbo.prc_VMM_ReleaseLock(@ObjectID=09b94958-3298-47ea-831c-6d73bc4c81d9, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @LockType=Delete)", @14-02-2017 09:38:39 [+00:00:00.0156254] }, { Q:"dbo.prc_BTBS_HostGroup_GetByID(@ID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_BTBS_HostGroup_GetChildGroupIDs(@ID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_BTBS_CustomPropertyValue_GetCPValIDByObjID(ObjectID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_Host_GetByGroupID(@HostGroupID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_Host_GetByComputerName(@ComputerName=Server35.fqdn)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_HostVolume_GetIDsByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_BTBS_CustomPropertyValue_GetCPValIDByObjID(ObjectID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_HostDisk_GetIDsByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ST_StorageFileShareAssociation_GetShareIDsByServerIDAndType(@ServerID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, @ServerType=VMHost)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_GetHostFibreChannelHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_GetHostSASHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_GetHostInternetSCSIHbaByHostID(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_HostGPU_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_NetMan_InstalledVirtualSwitchExtension_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_GetHostFibreChannelVirtualSANByHostId(HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_RBS_GetRunAsAccountConsumer(RunAsAccountID=, ObjectID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_RBS_GetRunAsAccountById(RunAsAccountID=1daa42f2-6a8b-42af-bdc0-fb79d7b4c047)", @14-02-2017 09:38:39 [+00:00:00.0156241] }, { Q:"dbo.prc_RBS_UserRoleSharedObjectRelation_GetSharedListForObject(ObjectID=1daa42f2-6a8b-42af-bdc0-fb79d7b4c047)", @14-02-2017 09:38:39 }, { Q:"dbo.prc_ADHC_HostDisk_GetByHost(@HostID=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=63b6a56b-01d4-4ce8-87b5-90a801a10e30)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=4d9a5fbf-96d9-4be6-945a-ef3add3fef1d)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=962883e9-9aaf-49a7-9988-3308aa11a362)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ADHC_HostVolume_GetByHostDiskID(@HostDiskID=053bf4ba-be66-4b8d-8094-f64f66f63c07)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ST_GetStorageFileShareIdsByAssociatedVolumeID(@AssociatedVolumeID=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:40 [+00:00:00.0155805] }, { Q:"dbo.prc_ST_GetReplicationGroupIdByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_DR_GetVolumeSynchronizedByVolumeId(VolumeId=9c9ce20d-9ac6-4c1d-ac98-5a5c56d9da96)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_BTBS_HostGroup_GetByID(@ID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_BTBS_HostGroup_GetChildGroupIDs(@ID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_BTBS_CustomPropertyValue_GetCPValIDByObjID(ObjectID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_ADHC_Host_GetByGroupID(@HostGroupID=9db34b98-02c3-44e6-8470-b807682ce203)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_WLC_GetVMObjectIdByHostId(HostId=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, ObjectType=VM)", @14-02-2017 09:38:40 }, { Q:"dbo.prc_WLC_GetPhysicalObjectIdByTypeHostId(HostId=76ffbcab-c3e3-4ab2-b9ca-7e3a95f586f3, ObjectType=VirtualHardDisk)", @14-02-2017 09:38:40 }, { NQ:"dbo.prc_VMM_AcquireLock(@ObjectID=8b5d0f5d-e477-413e-a998-17a1a8ca2016, @ObjectType=VirtualHardDisk, @TaskID=5d7b4b4d-d67b-4ee4-b64b-eca4a345553f, @ReqLockType=Delete)", @14-02-2017 09:38:40 } } , conn = (SqlConnection#178664f) { State: Open Database: VirtualManagerDB DataSource: Server99\HYPERV PacketSize: 8000 ServerVersion: 13.00.1722 StatisticsEnabled: False WorkstationId: Server99 } }, constructionIdentity = (WindowsIdentity#2d1ebed) System.Security.Principal.WindowsIdentity }) in s:\btvsts\3125\private\product\engine\ADHC\Operations\RemoveHostSubtask.cs:line 460
    [s#1a6ba6] Task`1.SubtaskRun(this: (Task`1<RemoveHostTask>#[11]) { TBD }, TaskID: (guid) 5d7b4b4d-d67b-4ee4-b64b-eca4a345553f)
    [s#199c03] RefreshDriver`1.RefreshThreadFunction(this: (RefreshDriver`1<ProMomAlertReference>#[1]) { tbd }, obj: (WaitCallbackState<ProMomAlertReference>#18ade09) Microsoft.VirtualManager.Engine.RefreshDriver`1+WaitCallbackState[Microsoft.VirtualManager.Engine.MOMDataAccessLayer.ProMomAlertReference]) in s:\btvsts\3125\private\product\engine\Common\RefreshDriver.cs:line 566
    Full call stack from when the exception was thrown:   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlContext.ExecuteNonQuerySP(String spName, Action`1 parameterAdder, Action`2 afterExecutionAction)
       at Microsoft.VirtualManager.DB.VmmDbTransaction.Microsoft.VirtualManager.DB.IVmmDbTransaction.ExecuteSP(String spName, Action`1 parameterAdder)
       at Microsoft.VirtualManager.DB.ImageLibrary.VHDDBAccess.Remove(Guid vhdId, IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDB(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDBForHostRemove(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.<>c__DisplayClass9.<RunSubtask>b__4(IVmmDbTransaction _tx)
       at Microsoft.VirtualManager.DB.SqlContext.Transact(CallSite callSite, String connectionString, WindowsIdentity identity, Action`1 action)
       at Microsoft.VirtualManager.DB.TransactableDbContext.Microsoft.VirtualManager.DB.IVmmDbConnection.Transact(Action`1 action)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveHostTask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.DB.SqlContext.Connect(Action`1 action)
       at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
       at System.Threading.ThreadPoolWorkQueue.Dispatch()
    --- SqlException details -----------------
    System.Data.SqlClient.SqlException (0x80131904): The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_tbl_WLC_VHD_VHD". The conflict occurred
    The statement has been terminated.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
    ClientConnectionId:3ebba05f-c839-4aa3-bdb9-93b2bb73fcdd
    Error Number:547,State:0,Class:16
     Error = 547
    Index #0
    Source: .Net SqlClient Data Provider
    Number: 547
    State: 0
    Class: 16
    Server: Server99\HYPERV
    Message: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_tbl_WLC_VHD_VHD". The conflict occurred
    Procedure: prc_WLC_RemoveVHD
    Line: 16
    Index #1
    Source: .Net SqlClient Data Provider
    Number: 3621
    State: 0
    Class: 0
    Server: SERVER99\HYPERV
    Message: The statement has been terminated.
    Procedure: prc_WLC_RemoveVHD
    Line: 16
    --- Caller StackTrace -----------------
    FileName:; Method:ExecuteNonQuery(); lineNo:0; ilOffset:171.
    FileName:; Method:ExecuteNonQuerySP(); lineNo:0; ilOffset:25.
    FileName:; Method:Microsoft.VirtualManager.DB.IVmmDbTransaction.ExecuteSP(); lineNo:0; ilOffset:6.
    FileName:; Method:Remove(); lineNo:0; ilOffset:17.
    FileName:; Method:RemoveFromDB(); lineNo:0; ilOffset:72.
    FileName:; Method:RemoveFromDBForHostRemove(); lineNo:0; ilOffset:73.
    FileName:; Method:<RunSubtask>b__4(); lineNo:0; ilOffset:60.
    FileName:; Method:Transact(); lineNo:0; ilOffset:16.
    FileName:; Method:Microsoft.VirtualManager.DB.IVmmDbConnection.Transact(); lineNo:0; ilOffset:47.
    FileName:; Method:RunSubtask(); lineNo:0; ilOffset:927.
    FileName:; Method:Run(); lineNo:0; ilOffset:390.
    FileName:; Method:RunSubtask(); lineNo:0; ilOffset:1871.
    FileName:; Method:Run(); lineNo:0; ilOffset:390.
    FileName:; Method:Connect(); lineNo:0; ilOffset:31.
    FileName:; Method:SubtaskRun(); lineNo:0; ilOffset:305.
    FileName:; Method:RunInternal(); lineNo:0; ilOffset:121.
    FileName:; Method:Run(); lineNo:0; ilOffset:9.
    FileName:; Method:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem(); lineNo:0; ilOffset:53.
    FileName:; Method:Dispatch(); lineNo:0; ilOffset:164.


    -------- exception.InnerException.ToString() -------
    System.Data.SqlClient.SqlException (0x80131904): The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_tbl_WLC_VHD_VHD". The conflict occurred
    The statement has been terminated.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
    ClientConnectionId:3ebba05f-c839-4aa3-bdb9-93b2bb73fcdd
    Error Number:547,State:0,Class:16


    -------- exception.InnerException.Data --------
    HelpLink.ProdName = Microsoft SQL Server

    HelpLink.ProdVer = 13.00.1722

    HelpLink.EvtSrc = MSSQLServer

    HelpLink.EvtID = 547

    HelpLink.BaseHelpUrl = http://go.microsoft.com/fwlink

    HelpLink.LinkId = 20476


    ----------------------------------------------------
    --------------- exception.StackTrace ---------------
    ----------------------------------------------------
    at Microsoft.VirtualManager.DB.SqlRetryCommand.ExecuteNonQuery()
       at Microsoft.VirtualManager.DB.SqlContext.ExecuteNonQuerySP(String spName, Action`1 parameterAdder, Action`2 afterExecutionAction)
       at Microsoft.VirtualManager.DB.VmmDbTransaction.Microsoft.VirtualManager.DB.IVmmDbTransaction.ExecuteSP(String spName, Action`1 parameterAdder)
       at Microsoft.VirtualManager.DB.ImageLibrary.VHDDBAccess.Remove(Guid vhdId, IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDB(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.ImageLibrary.VHD.RemoveFromDBForHostRemove(IVmmDbTransaction tx)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.<>c__DisplayClass9.<RunSubtask>b__4(IVmmDbTransaction _tx)
       at Microsoft.VirtualManager.DB.SqlContext.Transact(CallSite callSite, String connectionString, WindowsIdentity identity, Action`1 action)
       at Microsoft.VirtualManager.DB.TransactableDbContext.Microsoft.VirtualManager.DB.IVmmDbConnection.Transact(Action`1 action)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveGuestAgentsFromHostSubtask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.Adhc.RemoveHostTask.RunSubtask(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
       at Microsoft.VirtualManager.DB.SqlContext.Connect(Action`1 action)
       at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)


    ----------------------------------------------------
    ------------- StackTrace from handler --------------
    ----------------------------------------------------
    This is the call stack from where the exception was caught, not where it was thrown.
    at Microsoft.VirtualManager.Utils.Diagnostics.WatsonReport.WriteReportTextFile(TextWriter reportFile)
       at Microsoft.VirtualManager.Utils.Diagnostics.WatsonReport.Send()
       at Microsoft.VirtualManager.Utils.Diagnostics.WatsonExceptionReport.Send()
       at Microsoft.VirtualManager.Utils.Diagnostics.WatsonCenter.ReportException(Exception e, WERReportOptions options, String& localReportPath)
       at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
       at System.Threading.ThreadPoolWorkQueue.Dispatch()

    Error (802)
    The 'ISO' 'vmguest' is already in use by another 'ISO'. (Without any specific path/name/how should I track this down?)


    Recommended Action
    Wait for the object to become available, and then try the operation again.(NO, I was trying to install the host again and it fails and uninstall itself again on the target host machine, and I cannot use the local agent from DVD media where I installed SCVMM 2016 because it is not CU2 and SCVMM 2016 Standard CU2 does not provide a manuel reinstallation option where I can access a specific share and install the correct version agent.

    How do I remove Server35?


    • Edited by SysadminEU Wednesday, February 15, 2017 8:33 AM
    Tuesday, February 14, 2017 9:59 AM

All replies

  • Hello,

    Based on the error messages below, VMM can't connect to the database, since the SQL Server is not running.

    Please check and ensure the SQL Server is running. You can check it from the Services management tool.

    Microsoft.VirtualManager.DB.CarmineSqlException: Unable to connect to the VMM database because of a general database failure.
    Ensure that the SQL Server is running and configured correctly, then try the operation again.

    In addition, you may check the most recent update version Update Rollup 2 for SCVMM 2016

    More information, please see the link below.
    https://support.microsoft.com/en-us/help/3209586/update-rollup-2-for-system-center-2016-virtual-machine-manager

    Best regards,
    Andy Liu

    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Wednesday, February 15, 2017 3:06 AM
  • Hello,

    Based on the error messages below, VMM can't connect to the database, since the SQL Server is not running.

    Please check and ensure the SQL Server is running. You can check it from the Services management tool.

    Microsoft.VirtualManager.DB.CarmineSqlException: Unable to connect to the VMM database because of a general database failure.
    Ensure that the SQL Server is running and configured correctly, then try the operation again.

    In addition, you may check the most recent update version Update Rollup 2 for SCVMM 2016

    More information, please see the link below.
    https://support.microsoft.com/en-us/help/3209586/update-rollup-2-for-system-center-2016-virtual-machine-manager

    Best regards,
    Andy Liu

    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    The SQL server is running, otherwise the console would not function. I am still on CU1 :-( 

    I will update to CU2 later on and post the result.

    Wednesday, February 15, 2017 8:29 AM
  • This is a problem in Update Rollup 2.1 as well.
    /Lars

    Thursday, April 13, 2017 7:12 AM
  • Leave the host in for some hours after it has been removed from a cluster / emptied for VM's - it seems like VMM need some time before the host can be deleted.

    I successfully deleted a host that preciously failed with the above error.

    On occasions the same procedure has been successful immediately after removal  from cluster.

    /Lars

    • Proposed as answer by Willy Moselhy Tuesday, May 23, 2017 11:50 AM
    Thursday, April 13, 2017 12:44 PM
  • It is deleted and I am happy.
    Tuesday, May 23, 2017 7:43 AM
  • This has happened to us, but unfortunately it did not magically fix itself.

    Any ideas?

    Wednesday, July 12, 2017 4:35 AM
  • I managed to get this working for SCVMM 2016 Update Rollup 3.

    Before running the below TSQL:

    - Stop the "System Center Virtual Machine Manager"

    - Backup your VirtualManagerDB database

    - I take no responsibility for breaking your VMM if you run the TSQL.





    USE VirtualManagerDB; 
    go

    -- Replace the HostID = ' ' with your HostID
    DELETE FROM tbl_ADHC_VirtualNetwork WHERE HostID IN (SELECT HostID FROM tbl_ADHC_Host WHERE HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostVolume WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostDisk WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostToProcessorCompatibilityVectorMapping WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostSASHba WHERE SASHbaID IN (SELECT HbaID FROM tbl_ADHC_HostBusAdapter WHERE HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostBusAdapter WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_PhysicalObject WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_NetMan_InstalledVirtualSwitchExtension WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_Host WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_VNetworkAdapter WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')

    -- These tables have the "HostID" column, but 0 rows were affected for me when i run these.
    DELETE FROM tbl_ADHC_HostDisk WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostFibreChannelVirtualSAN WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostGPU WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostToProcessorCompatibilityVectorMapping WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_HostVolume WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_UnmanagedHost WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_VirtualNetwork WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_ADHC_VmwResourcePool WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_DR_ProtectionUnit_HostRelation WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_NetMan_InstalledVirtualSwitchExtension WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_PhysicalObject WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_VMDeploymentConfig WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_VMToNonPossibleOwnerHosts WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_VMToPreferredOwnerHosts WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')
    DELETE FROM tbl_WLC_VObject WHERE (HostID = '3FDC3880-B7E1-4FF5-8EF8-AA1A921E4263')


    -- Replace ClusterID = ' ' with your ClusterID.
    DELETE FROM tbl_ADHC_ClusterDisk WHERE (ClusterID = 'DA6C0951-FAEC-4304-9B85-5960D875A12C')
    DELETE FROM tbl_ADHC_HostCluster WHERE (ClusterID = 'DA6C0951-FAEC-4304-9B85-5960D875A12C')

    -- These tables have the "ClusterUD" column, but 0 rows were affected for me when i run these.
    DELETE FROM tbl_ADHC_UnmanagedHost WHERE (ClusterID = 'DA6C0951-FAEC-4304-9B85-5960D875A12C')
    DELETE FROM tbl_ADHC_UnmanagedHostCluster WHERE (ClusterID = 'DA6C0951-FAEC-4304-9B85-5960D875A12C')
    Wednesday, July 12, 2017 10:53 PM
  • I wanted to reply to this, as it is related to orphaned physical objects in the tbl_WLC_PhysicalObject table.  Somehow, if there is a clusterfailover or similar, the table is duplicate some disks, which may then be deleted.

    A simple fix seems to be:

    DELETE FROM [tbl_WLC_PhysicalObject] WHERE [HostId]='<HostGUID>'

    This will let you successfully remove the host from VMM.  If the VM's no longer exist, I would recommend being more selective about what you delete as VMM should delete them automatically, this is only for ophaned VM Objects that are still active elsewhere on a cluster.


    ----------------------------------------- Dan Sheppard

    • Proposed as answer by John McNair Friday, February 9, 2018 11:27 AM
    Tuesday, August 29, 2017 5:25 PM
  • I wanted to reply to this, as it is related to orphaned physical objects in the tbl_WLC_PhysicalObject table.  Somehow, if there is a clusterfailover or similar, the table is duplicate some disks, which may then be deleted.

    A simple fix seems to be:

    DELETE FROM [tbl_WLC_PhysicalObject] WHERE [HostId]='<HostGUID>'

    This will let you successfully remove the host from VMM.  If the VM's no longer exist, I would recommend being more selective about what you delete as VMM should delete them automatically, this is only for ophaned VM Objects that are still active elsewhere on a cluster.


    ----------------------------------------- Dan Sheppard

    This query worked for me.

    To find the value for HostID, I ran the following in VMM PowerShell:

    (Get-SCVMHost <HostName>).ID

    I was then able to remove the host.

    Friday, February 9, 2018 11:30 AM
  • I actully figured it out, and the reason was "Disk Queue Length" got so high that I had to reconfigure the MSA2040 to use Raid10.

    I am on Update 5, and I dont have the issue anymore.

    Saturday, June 23, 2018 11:27 AM