none
0x80070003 error during tape backup RRS feed

  • Question

  • Gents, I tried to backup to tape Windows 2008 R2 System State/Bare Metal Recovery items and see the following error:

    Triggering tape backup failed:

    Error 2033: DPM encountered an error while performing an operation for D:\WindowsImageBackup\ on dpm-server

    Error details: The system cannot find the path specified (0x80070003)

    Recommended action: 1) Refer to the detailed error code in the description above. Retry this operation after the issue has been fixed.

    2) If the error was due to insufficient resources, then it could be a transient failure and you should retry this operation after some time.

     

     I open R2 server in DPM 2010 console, choose System State/Bare Metal Recovery items on Recovery Tab -> (right click) Create recovery point -> Long term tape protection.

    Consistency check is OK. I did it again, no errors.

    I see some topics with the same error in this forum, but I don't see solution.

    I checked remote R2 server, D:\WindowsImageBackup\ doesn't exist. But MTA share exists.

    I see this issue with on R2 servers. These R2 servers were installed some time ago (1 month). Backup features were added after DPM agent installation.

    Two another R2 servers don't have this issue and tape backup works fine for System State/Bare Metal Recovery. These servers were installed approx 6 moths ago.

     

     

    Can anyone explain, what is wrong and how to fix that ?

     

    Thanks


    mcse^4
    • Edited by Alternate Wednesday, October 12, 2011 4:48 PM
    Wednesday, October 12, 2011 4:46 PM

Answers

  • I tried to restore and see the following error:

    DPM encountered an error while performing an operation for D:\WindowsImageBackup\ on dpm-server (ID 2033)

    I can't restore any recovery point of these bad servers.

     

    I opened 'View details', Replica Path is existing:

     

    How to fix that ?

    ps. There is only 1% of free space on DPM disks (3 x 1Gb).

     

    Thanks


    mcse^4
    Thursday, October 13, 2011 10:05 AM

All replies

  • Hi,

    Have you tried restoring that BMR backup to a network share to see if it restores OK ?    I suspect the problem might be that the link to the replica volume is missing - if you highlight the BMR data source in the PG, then click on "Click to view details"  link - is that dialog blank ?


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, October 12, 2011 8:24 PM
    Moderator
  • I tried to restore and see the following error:

    DPM encountered an error while performing an operation for D:\WindowsImageBackup\ on dpm-server (ID 2033)

    I can't restore any recovery point of these bad servers.

     

    I opened 'View details', Replica Path is existing:

     

    How to fix that ?

    ps. There is only 1% of free space on DPM disks (3 x 1Gb).

     

    Thanks


    mcse^4
    Thursday, October 13, 2011 10:05 AM
  • Hi,

    For some reason, DPM believes that the systemstate backup was taken from D:\WindowsImagebackup at one time.  Is this server a member of a cluster ?

    <snip>
    I checked remote R2 server, D:\WindowsImageBackup\ doesn't exist.
    >snip<

    Can you verify the present configuration of where DPM is storing the WindowsImageBackup by looking the configuration file as noted in this technet article.   If you make a change, please stop protection of the systemstate, delete replica, and reprotect to see if that clears up the problem.

    http://technet.microsoft.com/en-us/library/bb809015.aspx


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, October 29, 2011 4:28 PM
    Moderator
  • No, this server is not a member of cluster.

    I deleted these members, all data and recreated Protection groups. The issue is gone.

    !!! All backups of these members were deleted, it's really nice. It's a nice product, which reported about good backups, but you will know about it when you will restore servers ;-/

     


    mcse^4
    Monday, October 31, 2011 9:55 AM
  • Hi,

    I think the issue is that BMR protection does not use PSDatasourceConfig.xml for backup, but it effects restores if it is wrong, IE: points to a drive that is not present.  I will get this issue documented.  If you needed to get the restore done, we could have worked with you to get the data off the DPM replica.

    Periodic test restores should be a part of your backup strategy to help discover issues like this.

    Sorry for the inconvenience that issue has caused.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, October 31, 2011 7:07 PM
    Moderator
  • Mike, I've got the same issue on another R2 server. Monthly tape backup is failed and I can't restore System State of this server.

    How to fix that ?

    Thanks


    mcse^4

    • Edited by Alternate Monday, November 7, 2011 1:17 PM
    Monday, November 7, 2011 1:16 PM
  • Gents, is there any news ?

     


    mcse^4
    Tuesday, November 15, 2011 10:53 AM
  • Hi,

    Is the root cause the same - the PSDatasourceConfig.xml points to a non-existent volume on the protected server ?

    BMR volume on the DPM server is really locked down from a security standpoint, so to navigate to the replica volume so you can copy out the BMR backup use this procedure.

    1)      Download psexec.exe from www.sysinternals.com

    2)      Run PSEXEC -s cmd.exe   (this will switch window to system context)

    3)      Afterwards - type WHOAMI it should return:

                      nt authority\system

    4)  Now CD to the replica volume under program files\Microsoft DPM\DPM\Volumes\replica\...

    5)  Copy the BMR backup to a network location and share it out to restore from Winre.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, November 15, 2011 5:28 PM
    Moderator
  • Mike,

    Thank for answer. I can copy backup to another folder, but I would like to fix this issue.

    At the moment monthly tape backup is not done and I can't restore server for specific date.

    I can delete this server from Protection Group and add again, but will lose all daily backups for ~20 days (short protection).

    Is there way to fix it ?

    Why this issue happens ?

     


    mcse^4
    Wednesday, November 16, 2011 10:41 AM
  • Hi,

    The only "known" cause for ths issue is that the PSDatasourceConfig.xml on the protected server points to an invalid drive.  If that is not the case, please open a support incident with us so can investigate the issue further.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, November 16, 2011 2:44 PM
    Moderator
  • I see the following lines in PSDatasourceConfig.xml:

    - <DatasourceConfig>
      <WriterId>8c3d00f9-3ce9-4563-b373-19837bc2835e</WriterId>
      <Version>01.00.0000.00</Version>
      <VssWriterInvolved>false</VssWriterInvolved>
      <LogicalPath>Computer</LogicalPath>
      <ComponentName>System Protection</ComponentName>
      <FilesToProtect>C:\WindowsImageBackup\*</FilesToProtect>
      <Size>51200</Size>
      <UseDRWithCC>true</UseDRWithCC>
    </DatasourceConfig>
     
    C:\ is system and boot volume. But C:\WindowsImageBackup doesn't exist.
     
    We use DPM by Gold Cert Partner program, not Volume Licensing.
    I can't open a support case.
     
     

    mcse^4
    Wednesday, November 16, 2011 3:31 PM
  • Hi,

     

    OK - then this is a condition we have not seen before, and needs additional troubleshooting that cannot be performed in a forum setting.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, November 17, 2011 3:44 PM
    Moderator
  • If you need any help or more details, please let me know. I'd like to fix this issue as soon as possible.

    I noticed that this issue happens when we forgotten to insert a free tape into tape library. We can do it with 1-3 days delay. If it can help...

    Thanks


    mcse^4
    Thursday, November 17, 2011 4:38 PM
  • dpm 2010 last patch, win 2008 R2 stand alone server,colocation enabled , 16 tapes autoloader, no problems of space like the error says, no cluster, same issue of Alternate

    my tape backup failed after 2 days i continued it from "job waiting tape" , and seems always when we forgot to insert free tapes , for example on weekend but we have time to do that for registry changes with 1-5 delay. here are  3 email alerts, always these 3 email, same PG, the longer (500GB , about 8 tapes), please see also the time of 3 alerts:

     

    The following alert occurred at 22/11/2011 13:07:23.

    Status: Active
    Description: Backup to tape failed.
    DPM failed to access the volume \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy3825\ on

    . This could be due to
    1) Cluster failover during backup or

    2) Inadequate disk space on the volume

    For more information, open DPM Administrator Console and review the alert details in the Monitoring task area.

    The following alert occurred at 22/11/2011 13:07:25.

    Status: Active
    Description: Backup to tape failed.
    DPM encountered an error while performing an operation for \\?\Volume{5c405246-170b-11dd-a996-806e6f6e6963}\ on

    For more information, open DPM Administrator Console and review the alert details in the Monitoring task area.

    The following alert occurred at 22/11/2011 13:07:57.

    Status: Active
    Description: Backup to tape failed.
    DPM encountered an error while performing an operation for \\?\Volume{f4074a14-c556-11dc-8773-000c29c7187e}\ on

    For more information, open DPM Administrator Console and review the alert details in the Monitoring task area.

    after that,  i see in managment menu,  expire tapes of last job beacuse tape job failed,  but if click on RUN JOB AGAIN i have always error that "no recovery point is present" , that is true, beacuse my RP can have only 1 week retention for the limit of vss data (64 very low  , 2012 fix this low limit?) , so ALL TAPE JOBS, long tape jobs of days is always losen, this is the real tape issue of dpm 2007 and 2010:

    So the problem is the error , but the real problem is the NO tape job resume chance in colocation, i think (also i can't restart my dpm server with job waiting tape message, but this is less important)

    bye and thanks

     

    Tuesday, November 22, 2011 11:23 PM
  • Lorenzo, can you please open a support case in support ?

    Thanks


    mcse^4
    Wednesday, November 23, 2011 9:59 AM
  • OK i do that bye
    Thursday, November 24, 2011 1:01 AM
  • sorry how can i open a support case for this case?

    bye

    Thursday, November 24, 2011 1:04 AM
  • If you bought DPM by Open Volume Subscription, you can open any number cases via web interface.

    http://support.microsoft.com , see at the bottom "Contact a Support Professional by Email, Online, or Phone"

    Software Assurance, enter Software Assurance code and email for that, choose 1 day - email.

     


    mcse^4
    Thursday, November 24, 2011 9:23 AM
  • Hi Mike,

    I have exactly the problem your are describing, I get the same error message '0x80070003' tape jobs fails for system state.
    When I look at the replica path I only see a blank dialog

    How could I fix that?

     

    Thursday, December 1, 2011 4:16 PM
  • Hi

    I think the issue is that BMR protection does not use PSDatasourceConfig.xml for backup, but it effects restores if it is wrong, IE: points to a drive that is not present. 

    Check to see if the PSDatasourceConfig.xml is pointing to a non-existent drive letter, and if so, change it to a valid drive letter.

    1. On the protected computer, open PSDatasourceConfig.xml in an XML or text editor. PSDatasourceConfig.xml is typically located at install path\Program Files\Microsoft Data Protection Manager\DPM\Datasources.

    2. Under the System Protection section as shown below - Change the <FilesToProtect> value from present invalid drive letter IE: F: to the desired new locations drive letter IE: C: (do not change the folder name)


    <ComponentName>System Protection</ComponentName>
    <FilesToProtect>C:\WindowsImageBackup\*</FilesToProtect>


    3. Save the file.

    4. Stop BMR protection and delete the replica, then reprotect the servers BMR workload.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    • Proposed as answer by Henti Wednesday, July 18, 2012 1:58 PM
    Thursday, December 1, 2011 5:51 PM
    Moderator
  • Mike, Do you mean that DPM server should have the same drive letter as in the PSDatasourceConfig.xml or backup will be failed ?

     

    I have issue with sql server. There is no enough free space on disk C:\ for full backup. I tried to change to C:, but backup was failed with error "No free space".

    At the moment FilesToProtect is pointing to drive T:

    <FilesToProtect>T:\WindowsImageBackup\*</FilesToProtect>

    But DPM server doesn't have drive T:\.

     

     


    mcse^4

    • Edited by Alternate Friday, December 2, 2011 9:47 AM
    Friday, December 2, 2011 9:46 AM
  • Hi,

    I'm sorry, I thought your problem was for BMR like the others.  I have not seen this before for systemstate - please open a support case for complete investigation.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, December 2, 2011 9:24 PM
    Moderator
  • Installed DPM2012. I see the same issues as described in the first post.

    Gents, WHEN it will be fixed ?


    mcse^4

    Tuesday, July 3, 2012 5:19 PM
  • Hi,

    As stated earlier in this post, we cannot troubleshoot this particular problem in a forum, please open a support incident so we can investigate your specific problem.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Wednesday, July 4, 2012 12:24 PM
    Moderator
  • Hello,

    had the same issue and for me Mikes solution was helpful. Changed the nonexisting path to an exisiting , stopped protection, deleted replicas, reprotect, everything working.

    Reason for the wrong path was the server being a part of an SQL Cluster, and initially it seems to happen that the largest drive is taken for the "filestoprotect" entry. If the cluster node is changed this drive is no longer accessible. The automatic choice of this path should better not include cluster volumes ...

    Thank you.

    Greetings, Frank

    Wednesday, July 18, 2012 2:05 PM
  • Hi,

    I had similar issue (Error details: The system cannot find the path specified (0x80070003)) during tape backup for BMR protection, and Mike solution did solve the problem. Thanks.

    When I looked into PSDatasourceConfig.xml file, FileToProtec was pointing to an non-existing drive:<FilesToProtect>F:\WindowsImageBackup\*</FilesToProtect>

    However, can someone explain me how does it come to have the FileToProtec  pointing suddenly to an non-existing drive letter, while tape backup was working fine before for BMR protection?

    Why does it suddenly stop working?

    Regards,

    Evan

    Thursday, August 7, 2014 9:34 AM
  • Hi,

    This usually occurs when protecting a cluster node and the DPM agent is installed when it owns the clustered disks.  DPM agent install selects the drive with the most free space which in many cases is the clustered volume. later that volume is failed over to another node and suddenly the drive letter is not valid.

    This issue was fixed in DPM 2012 UR3.

    Issue 7

    A bare metal recovery fails in certain situations.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, August 7, 2014 3:49 PM
    Moderator
  • Hi,

    I have the same issue on two machines in a PG that contains three, the other machine is fine. Every Saturday when the Backups to tape are run I get a Backup to tape failed for both Bare Metal Recovery and System State. Interestingly, the error details are slightly different for the two machines, what I get on one machine is "DPM encountered an error while performing an operation for D:\WindowsImageBackup\ on escalade.network.co.uk(ID 2033 Details: The system cannot find the path specified (0x80070003))"

    However, the path that has been talked about is there in the PSDatasourceConfig.xml file <FilesToProtect>D:\WindowsImageBackup\*</FilesToProtect>, and that path on the physical drive.

    The thing that's different on the other machine is the path is different ie C: and there is no C:\WindowsImageBackup on the machine at all but I get the same message, so they both can't be right.

    What's really interesting is that on my machine that's working fine it also has a path C:\WindowsImageBackup in the PSDatasourceConfig.xml file, and that doesn't appear on the physical drive at all!!!

    What I've generally done to get past the problem, is take SS and BMR out of the protection group and then add it back in and the problem goes away until the next week when it returns with the same two machines having the same problem. Clearly there's something awry, but I cannot fathom out what it is. Can anybody point me in the right direction?

    Thanks

    Monday, July 13, 2015 3:25 PM
  • As I can see, this issue is repeated in some cases.

    I opened a support case and MS support helped me fixing this issue.

    I haven't seen this issue again with the latest version of DPM (keep fingers crossed ;)

    I fixed this issue a lot of time ago and there is only a few details.

    You can use that solution at your own risk.

    Regarding the drive letter in the xml file:

    DPM uses the drive with the most free space at the time the agent was installed, but does not ever change it after that.

    The workaround below is broken into two part. Part “A” has workaround for making the newer BMR backups recoverable. The part “B” would make even the older BMR backups as recoverable.

    Please create a backup about the DPMDB first!

    http://technet.microsoft.com/en-us/library/ff399388.aspx


    On the DPM server, open an elevated command prompt window, type cd <system drive>:\Program Files\Microsoft DPM\DPM\bin, and then type DPMBackup.exe -db and press ENTER.

    Then follow these steps:

    -- A.

    -- 1. Run the below SQL query on the DPMDB database for the particular Protection Server with Server Name (e.g. below whms0135) and get the Physical Replica Id and DatasourceId.

    Select D.PhysicalReplicaId, A.DataSourceId, C.DatasourceSequenceNumber, A.ProtectedGroupId, A.DataSourceName, A.ApplicationPath

    from dbo.tbl_IM_DataSource A

    join tbl_AM_Server B

          on A.ServerId = B.ServerId

    join tbl_PRM_DatasourceConfigInfo C

          on A.DataSourceId = C.DatasourceId 

    join tbl_PRM_LogicalReplica D

          on D.DatasourceId=A.DatasourceId

    where B.ServerName like '%whms0135%'and

    A.AppId = '8C3D00F9-3CE9-4563-B373-19837BC2835E'

    -- 2. Open a Command window as Administrator and run the below command and get the volume drive letter based directory. For e.g. if the directory

    --    contains D-Vol then note down the "D" drive. There should be only Drive letter based directory present. If there are

    --    multiple then see which one has WindowsImageBackup folder underneath and note that drive letter and remove other directories. Only

    --    one of them should be having WindowsImageBackup folder underneath. In case of doubt please contact PG before proceeding.

    dir "%ProgramFiles%\Microsoft System Center 2012\DPM\DPM\Volumes\Replica\Non Vss Datasource Writer\vol_<PhysicalReplicaId>\<DatasourceId>\Full"

    -- 3. Use that Drive letter noted above and put into the PSDatasourceConfig.xml on the corresponding PS Server under FilesToProtect node for “System    

          Protection”. Make sure the volume with the drive letter exists.

    -- 4. Restart the DPMRA service

    -- 5. Take an adhoc BMR backup.

    -- 6. Try recovery from the latest BMR backup which should succeed.

    -- B. For making the older BMR backup (which were failing during BMR recovery)to be also recoverable

    -- 1. Run the following SQL query after substituting the values for DatasourceId

    --    and the drive letter as fetched above in the step A.1.

    update tbl_RM_RecoverableObjectFileSpec

    Set Path = '<DriveLetter>:\WindowsImageBackup\'

    where RecoverableObjectFileSpecid in

    (

    Select B.RecoverableObjectFileSpecId

    from dbo.tbl_RM_RecoverableObject A

    join dbo.tbl_RM_RecoverableObjectFileSpec B

    ON B.RecoverableObjectId = A.RecoverableObjectId

    join dbo.tbl_RM_RecoverySource C

    ON B.DataSetId=C.DataSetId

    where C.IsValid = 1 and

    B.DatasetId  in

          (Select B.DataSetId

                from dbo.tbl_PRM_LogicalReplica A

                join dbo.tbl_RM_ReplicaDataset B

                on A.ReplicaId=B.ReplicaId

                where A.DataSourceId = '<DatasourceId>'

          )

    )

    -- 2. Restart the DPM UI and try out the BMR recovery from the older BMR backup that was not working and it should work now.


    mcse^4


    mcse^4

    Monday, July 13, 2015 3:59 PM
  • Hi,

    Thanks for the info. One thing you've said which got a bell ringing in my head. The two machines that I'm having issues with, at least one has been rebuilt from scratch (my colleague can't remember what he did with the other but it could have been). What I can say with cartainty, is the one that was rebuilt was rebuilt with the same name. Could this have implications?? If it does, would removing it from the protection Group, renaming the machine and then re-inserting in the protection group fix the issue? Just a thought.

    Monday, July 20, 2015 3:51 PM