none
Backup to tape keeps failing RRS feed

  • Question

  • Type:	Tape backup 
    Status:	Failed
    Description:	The operation failed because of a protection agent failure. (ID 998 Details: The device is not connected (0x8007048F))
    	More information
    End time:	11/29/2015 1:39:21 AM
    Start time:	11/28/2015 11:25:10 PM
    Time elapsed:	02:14:11
    Data transferred:	382,065.56 MB
    Cluster node	-
    Source details:	J:\
    Protection group:	SQL Backup Group (SQL2K5/SQL2K8) Lib Vols
    Library:	Hewlett Packard MSL G3 Series library  (x64 based)
    Tape Label (Barcode):	SQL Backup Group (SQL2K5/SQL2K8) Lib Vols-LT-1Weeks-Copy0-00000165 (ACA315L5)

    So the job will run for about 2 hours then fail. The only way to get it to finish is to retry the job. Annoying but gets the job done. This has started recently. Starting to wonder if my tapes are going bad or something. Here is the error message Im getting (above). I searched the logs here is what I found

    ---------- MSDPM108.ERRLOG
    18A0 1F6C 11/29 06:37:33.361 27 CommonErrorHandler.cs(165)  17BCFF42-9194-4BA0-89B8-5A0F9877A96A WARNING AgentStatus[MTAForWrite] - (CommandID=MTAPerformIO, StatusReason=Error) failed with HRESULT 0x8007048F, error -2147023729.
    18A0 1F6C 11/29 06:37:33.428 27 CommonErrorHandler.cs(165)  17BCFF42-9194-4BA0-89B8-5A0F9877A96A WARNING AgentStatus[MTAForWrite] - (CommandID=MTAPerformIO, StatusReason=Error) failed with HRESULT 0x8007048F, error -2147023729.

    Any ideas ? This is an HP LTO5 MSL 2024 Library. Latest Drive/Chassis firmware. Latest HP Driver.

    ** Update, ran ERR.EXE -2147023729, I do see some "Out of memory" returns. I wonder, if another job is running and the machine gets low on memory while doing the tape backup? I know windows has memory pressure protection. Wouldn't I see an event ID for that? Currently I'm seeing event ID 15 'The device, \Device\TapeDrive0, is not ready for access yet.' Source 'hplto'

    ** Update Again ( you guys are too slow) :)

    Found this from HP http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=mmr_kc-0100617 going to give it a shot!

    • Edited by Mike_Kubi Monday, November 30, 2015 6:33 PM
    Monday, November 30, 2015 3:12 PM

Answers

  • For all this that might be following, so far so good with the registry fix above!
    • Proposed as answer by Michael Seidl (MVP)MVP Wednesday, December 2, 2015 11:04 AM
    • Unproposed as answer by Mike_Kubi Wednesday, December 9, 2015 5:26 PM
    • Marked as answer by Mike_Kubi Wednesday, December 9, 2015 5:26 PM
    Tuesday, December 1, 2015 4:20 PM

All replies

  • 0590	3798	11/25	18:08:51.072	28	FsmBlock.cs(178)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	Backup.MMBackupLoop : MTAPerformIO, StatusReason = Error (StatusCode = -2147023729, ErrorCode = GenericAgentFailure, workitem = 286c45fa-e389-4319-9ab1-da9d518e343e)
    0590	3798	11/25	18:08:51.072	28	FsmBlock.cs(178)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	Response: <?xml version="1.0"?>
    0590	3798	11/25	18:08:51.072	28	FsmBlock.cs(178)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	<Status xmlns="http://schemas.microsoft.com/2003/dls/StatusMessages.xsd" StatusCode="-2147023729" Reason="Error" CommandID="MTAPerformIO" CommandInstanceID="729c31e8-7391-4bca-9add-083b6e5ffa88" GuidWorkItem="286c45fa-e389-4319-9ab1-da9d518e343e" TETaskInstanceID="c57e9ef3-238f-4a44-a06a-a8c93b4ace79"><ErrorInfo xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd" ErrorCode="998" DetailedCode="-2147023729" DetailedSource="2"/></Status>
    0590	3798	11/25	18:08:51.129	27	CommonErrorHandler.cs(165)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	AgentStatus[MTAForWrite] - (CommandID=MTAPerformIO, StatusReason=Error) failed with HRESULT 0x8007048F, error -2147023729.
    0590	3798	11/25	18:08:51.129	27	CommonErrorHandler.cs(265)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Unmapped agent error code = GenericAgentFailure
    0590	3798	11/25	18:08:51.129	27	CommonErrorHandler.cs(76)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	error GenericAgentFailure has been already supplied - do not override it with RmGenericError
    0590	3798	11/25	18:08:51.145	28	MMBackupLoop.cs(1091)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	MTAPerformIO failed, taskError=GenericAgentFailure
    0590	3798	11/25	18:08:51.145	28	MMBackupLoop.cs(1253)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	MMBackupLoop Failed, taskError=GenericAgentFailure
    0590	3798	11/25	18:08:51.147	27	CommonErrorHandler.cs(165)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	AgentStatus[MTAForWrite] - (CommandID=MTAPerformIO, StatusReason=Error) failed with HRESULT 0x8007048F, error -2147023729.
    0590	3798	11/25	18:08:51.147	27	CommonErrorHandler.cs(265)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Unmapped agent error code = GenericAgentFailure
    0590	3798	11/25	18:08:51.147	27	CommonErrorHandler.cs(76)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	error GenericAgentFailure has been already supplied - do not override it with RmGenericError
    0590	3798	11/25	18:08:51.148	27	BackupLoop.cs(301)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	BackupLoop : MMBackupLoop Failure, error=GenericAgentFailure
    0590	3798	11/25	18:08:51.163	28	FsmBlock.cs(198)			WARNING	Backup.OnAbort.MMCancelBlock.LACancelBlock : Not an agentStatusMessage
    0590	3798	11/25	18:08:51.171	27	MTARequestor.cs(107)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Sending Command MTACancelAllSubTasks to MTAForWrite on server kdnap-util1.**EDITED**.pvt for workitem 286c45fa-e389-4319-9ab1-da9d518e343e
    0590	3798	11/25	18:08:51.181	27	FsmBlock.cs(167)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Backup.OnAbort(2).MTADeactivationBlock.MTACancelAllSubTasksBlock : MTACancelAllSubTasks, StatusReason = Unknown (StatusCode = 0, ErrorCode = Success, workitem = 286c45fa-e389-4319-9ab1-da9d518e343e)
    0590	3798	11/25	18:08:51.212	28	EventHelper.cs(504)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing DriveStateChanged event for DriveId=0ee832d7-2303-45cf-9490-a6f0289db882. IsEnabled=True, IsOffline=False
    0590	3798	11/25	18:08:51.212	02	EventManager.cs(88)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing event from TapeDrive.cs(319): DriveStateChanged, [DriveID=0ee832d7-2303-45cf-9490-a6f0289db882]
    0590	2980	11/25	18:09:18.324	01	TaskInstance.cs(364)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	Task: Message 'Manual:Wakeup' ignored since it was not meant for the current state 'Backup.OnAbort..LADismountDrivePending' for id=8e6b1e73-06ab-45a9-a217-18b586a1e53f
    0590	2980	11/25	18:09:52.365	28	EventHelper.cs(604)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing MediaOperationSucceeded event for MediaId=3fb5acf4-abc3-4f3b-9a78-19c005e8dfdd, Operation=Dismount
    0590	2980	11/25	18:09:52.365	02	EventManager.cs(88)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing event from Media.cs(1056): MediaOperationSucceeded, [MediaID=3fb5acf4-abc3-4f3b-9a78-19c005e8dfdd]
    0590	2980	11/25	18:09:52.373	28	EventHelper.cs(943)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing MediaStatusChangedEvent for libraryId=16810f19-9102-453d-89bb-03302635d427
    0590	2980	11/25	18:09:52.373	02	EventManager.cs(88)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing event from Media.cs(1400): MediaStatusChanged, [TaskID=c57e9ef3-238f-4a44-a06a-a8c93b4ace79]
    0590	2980	11/25	18:09:52.455	28	Media.cs(165)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Unreserving media 3fb5acf4-abc3-4f3b-9a78-19c005e8dfdd from task c57e9ef3-238f-4a44-a06a-a8c93b4ace79
    0590	2980	11/25	18:09:52.486	27	RARequestor.cs(188)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Sending Command RACancelAllSubTasks to RAForRead on server kdnap-util1.**EDITED**.pvt for workitem 2632dfa0-77eb-48be-beda-0ed31b0b9857
    0590	3360	11/25	18:09:52.645	27	BackupMachine.cs(2737)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	BackupMachine : DBCleanup on backup failure with errorCode=GenericAgentFailure
    0590	3360	11/25	18:09:52.645	27	BackupMachine.cs(2745)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	BackupMachine: Cleaning up ROlist for DatasetId [05891653-0e4f-4941-bc2e-2720c15322c2] on backup failure, will also mark Dataset as Invalid
    0590	32C0	11/25	18:09:52.671	18	fsutils.cpp(3710)			WARNING	Failed: Hr: = [0x80070002] : CreateFile failed for \\?\C:\Program Files\Microsoft DPM\DPM\Temp\c57e9ef3-238f-4a44-a06a-a8c93b4ace79\
    0590	3360	11/25	18:09:52.852	27	OperationTypeLock.cs(1043)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	In Unlock() of ReplicationOperationTypeLock, returning
    0590	3360	11/25	18:09:52.864	02	EventManager.cs(88)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing event from ServerAlert.cs(556): AlertStateChange, [ObjectId=e238e2f1-2126-454a-baec-d6ce13ae9e35]
    0590	3360	11/25	18:09:52.912	27	BackupMachine.cs(2596)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	BackupMachine : FAILURE - BACKUP, errorCode=GenericAgentFailure
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	Task Diagnostic Information - <?xml version="1.0" encoding="utf-16"?>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	<TaskExecutionContext>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmWriterId>00000000-0000-0000-0000-000000000000</PrmWriterId>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmDatasourceId>e238e2f1-2126-454a-baec-d6ce13ae9e35</PrmDatasourceId>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmReplicaDatasetId>4d7ade89-ccbd-4d4f-9879-966243e1b3d6</PrmReplicaDatasetId>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmLogicalReplicaId>3473a0b9-523e-44fb-88f9-1f358e365f4d</PrmLogicalReplicaId>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmShadowCopyId>1827ae83-65bd-4ae6-8849-0e671f45b916</PrmShadowCopyId>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <PrmActiveNodeName>kdnap-util1.**EDITED**.pvt</PrmActiveNodeName>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <TEVerb>ArchiveFromSC</TEVerb>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <TEErrorState>Backup.BackupLoopWaitState</TEErrorState>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  <TEErrorDetails>&lt;?xml version="1.0" encoding="utf-16"?&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	&lt;q1:ErrorInfo ErrorCode="998" DetailedCode="-2147023729" DetailedSource="2" ExceptionDetails="" xmlns:q1="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd"&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="drivename" Value="Hewlett Packard LTO Ultrium-5 drive" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="medialocationtype" Value="Drive" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="library" Value="Hewlett Packard MSL G3 Series library  (x64 based)" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="medialocationinfo" Value="Hewlett Packard LTO Ultrium-5 drive" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="mediabarcode" Value="ACA307L5" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="librarytype" Value="Tape Library" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	  &lt;q1:Parameter Name="medialabel" Value="SQL Backup Group (SQL2K5/SQL2K8) Lib Vols-LT-1Weeks-Copy0-00000164 (ACA307L5)" /&gt;
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	&lt;/q1:ErrorInfo&gt;</TEErrorDetails>
    0590	3360	11/25	18:09:52.919	01	TaskInstance.cs(800)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	WARNING	</TaskExecutionContext>
    0590	3750	11/25	18:09:52.921	03	timer.cpp(513)	[000000004C751148]		ACTIVITY	Shutting down timer thread.
    0590	3360	11/25	18:09:52.921	02	EventManager.cs(88)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	NORMAL	Publishing event from TaskInstance.cs(825): TaskStop, [TaskID=c57e9ef3-238f-4a44-a06a-a8c93b4ace79]
    0590	3360	11/25	18:09:52.929	01	TaskExecutor.cs(843)		C57E9EF3-238F-4A44-A06A-A8C93B4ACE79	FATAL	Task stopped (state=Failed, error=GenericAgentFailure; -2147023729; WindowsHResult), search "Task Diagnostic Information" for details.
    0590	3360	11/25	18:09:52.930	16	ActiveJob.cs(688)			WARNING	 Fail: Task 'c57e9ef3-238f-4a44-a06a-a8c93b4ace79' failed with error during execution.
    0590	3360	11/25	18:09:52.930	16	Task.cs(235)			NORMAL	 Changing task state from 'GenerateWorkplan' -> 'Failed' (c57e9ef3-238f-4a44-a06a-a8c93b4ace79)
    

    Monday, November 30, 2015 5:35 PM
  • For all this that might be following, so far so good with the registry fix above!
    • Proposed as answer by Michael Seidl (MVP)MVP Wednesday, December 2, 2015 11:04 AM
    • Unproposed as answer by Mike_Kubi Wednesday, December 9, 2015 5:26 PM
    • Marked as answer by Mike_Kubi Wednesday, December 9, 2015 5:26 PM
    Tuesday, December 1, 2015 4:20 PM
  • Its been awhile, no more errors the above is the fix!
    Wednesday, December 9, 2015 5:25 PM