none
DPM tape backup running for hours without any progress RRS feed

  • Question

  • Dear All,

    DPM tape backup running for hours without any progress.

    Troubleshooting done so far,

    -Have tried cancel the job

    result - did not gets cancel and after some 20 - 30 mins i see again the job is in progress.

    -Tried to stop the PG

    result - no progress it was just hung

    -Tried stopping the communication agent

    result - agent communication was disabled - But no change still the job was active.


    Thursday, August 6, 2020 7:08 AM

Answers

  • Hi,

    In order to forcefully stop the job, run below SQL query against the DPMDB. Follow below steps:

    1. Close the DPM Console.

    2. Take a fresh DPMDB backup.

    3. Run below SQL query against the DPMDB.

    4. Reopen the console and verify if job is now gone.

    BEGIN TRAN  
     
    -- mark replica as invalid if there was some operation happening on that replica  
     
    UPDATE tbl_PRM_LogicalREplica  
    SET Validity = 1 -- Invalid  
    WHERE OwnerTaskIdLock IS NOT NULL AND  
    Validity <> 5 AND -- ProtectionStopped  
    Validity <> 6 -- Inactive  
     
    -- Release all the locks held  
     
    UPDATE tbl_PRM_LogicalREplica  
    SET OwnerTaskIdLock = null,  
    Status=8  
     
    if (select COUNT(name) from tbl_AM_Agent where Name like 'DPM RA v2%') > 0  
    begin  
    exec sp_executesql N'UPDATE tbl_RM_ShadowCopy  
    SET ArchivetaskId = NULL,  
    RecoveryJobId = NULL'  
    end  
     
    UPDATE tbl_ARM_Datasource  
    SET Status = 0,  
    OwnerLockId = NULL  
     
    DELETE tbl_RM_DatasourceServerlock  
     
    DELETE tbl_RM_ShadowCopyLocks  
     
     
    -- Set All running tasks and jobs to failed  
     
    UPDATE tbl_TE_TaskTrail  
    SET ExecutionState = 3,  
    LastStateName = 'Failure',  
    StoppedDateTime = GetUtcDate()  
    WHERE ExecutionState NOT IN (2,3)  
     
    UPDATE tbl_JM_JobTrail  
    SET JobState= 'Failed',  
    EndDateTime = GetUtcDate()  
    WHERE jobstate= 'Execute' OR jobstate= 'Retire' OR jobstate = 'Cancel' or JobState = 'GenerateWorkplan' or JobState = 'warning' or JobState = 'Start'  
     
    -- unreserve resources held  
     
    UPDATE tbl_MM_Global_Media  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    UPDATE tbl_MM_Global_Drive  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    UPDATE tbl_MM_Global_IEPortResource  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    COMMIT TRAN  
    NOTE: This script will cancel all the running jobs. So please keep that in mind before running it. Thanks

    Regards
    Aayoosh Moitro


    Please remember to mark the replies as answers if they help and unmark them if they provide no help. This posting is provided "AS IS" with no warranties, and confers no rights.


    Thursday, August 6, 2020 7:19 AM

All replies

  • Hi,

    In order to forcefully stop the job, run below SQL query against the DPMDB. Follow below steps:

    1. Close the DPM Console.

    2. Take a fresh DPMDB backup.

    3. Run below SQL query against the DPMDB.

    4. Reopen the console and verify if job is now gone.

    BEGIN TRAN  
     
    -- mark replica as invalid if there was some operation happening on that replica  
     
    UPDATE tbl_PRM_LogicalREplica  
    SET Validity = 1 -- Invalid  
    WHERE OwnerTaskIdLock IS NOT NULL AND  
    Validity <> 5 AND -- ProtectionStopped  
    Validity <> 6 -- Inactive  
     
    -- Release all the locks held  
     
    UPDATE tbl_PRM_LogicalREplica  
    SET OwnerTaskIdLock = null,  
    Status=8  
     
    if (select COUNT(name) from tbl_AM_Agent where Name like 'DPM RA v2%') > 0  
    begin  
    exec sp_executesql N'UPDATE tbl_RM_ShadowCopy  
    SET ArchivetaskId = NULL,  
    RecoveryJobId = NULL'  
    end  
     
    UPDATE tbl_ARM_Datasource  
    SET Status = 0,  
    OwnerLockId = NULL  
     
    DELETE tbl_RM_DatasourceServerlock  
     
    DELETE tbl_RM_ShadowCopyLocks  
     
     
    -- Set All running tasks and jobs to failed  
     
    UPDATE tbl_TE_TaskTrail  
    SET ExecutionState = 3,  
    LastStateName = 'Failure',  
    StoppedDateTime = GetUtcDate()  
    WHERE ExecutionState NOT IN (2,3)  
     
    UPDATE tbl_JM_JobTrail  
    SET JobState= 'Failed',  
    EndDateTime = GetUtcDate()  
    WHERE jobstate= 'Execute' OR jobstate= 'Retire' OR jobstate = 'Cancel' or JobState = 'GenerateWorkplan' or JobState = 'warning' or JobState = 'Start'  
     
    -- unreserve resources held  
     
    UPDATE tbl_MM_Global_Media  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    UPDATE tbl_MM_Global_Drive  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    UPDATE tbl_MM_Global_IEPortResource  
    SET ReservationLevel = 0,  
    ReservationOwnerMMId = null  
     
    COMMIT TRAN  
    NOTE: This script will cancel all the running jobs. So please keep that in mind before running it. Thanks

    Regards
    Aayoosh Moitro


    Please remember to mark the replies as answers if they help and unmark them if they provide no help. This posting is provided "AS IS" with no warranties, and confers no rights.


    Thursday, August 6, 2020 7:19 AM
  • Thanks Aayoosh.

    It worked as a champ … 

    Thursday, August 6, 2020 10:25 AM