none
DPM 2012 R2 - ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)

    Question

  • Almost all of our backups are running and completing fine but we have 4 tape job's that will begin to backup then error out at about the same point all the time with the same Error Details (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)). All 4 tape job's that fail are on the same 2 clustered servers, but we do have 3 job's on the same server cluster that will complete. Any help would be greatly appreciated, Thanks.

    This is a list of our setup:

    Single DPM 2012 R2 server running on Windows 2012 R2 Datacenter

    2 clustered file servers running Windows 2012 R2 Standard

    Data Deduplication is enabled on both file servers

    Tape Library is Dell PowerVault TL2000 (LTO5)

    Disk storage is Dell PowerVault MD3200 direct attached with about 75TB

    Thursday, March 27, 2014 5:39 PM

All replies

  • Hi,

    Same problem here.

    Sometimes the backup to tape finished with the error id 2019 and sometimes with 3311.

    The specifications are approximatively the same:

    DPM 2012 R2 on Windows 2012 R2

    Tape Library Dell PowerVault 24T (LTO 4)

    Disk Storage EonStor with 10TB

    The backup to tape job process 10 servers and the problem appear always on the same server. A Windows 2012 R2 with data deduplication activated. In the past we already had this type of problem, our workaround is to backup the directory in place of the all disk.  In this case we lost the benefit of data deduplication for the backup but it works.  This time it's not possible to apply this workaround because the rehydrated data overtake our backup disk capacity :(

    It's the second problem with the backup of deduplicated volume.  The first one reported the backup to disk as failed and force a consistency check.  This problem was solved by Microsoft after 3 months with the last update.

    How can I explain to my boss that we must live without backup of critical datas during this lap of time ?  And now another problem to backup to tape....

    I seriously investigate other solutions to backup our Datacenter.  Veeam seems to work like a charm with all configuration.Since many years I push to work with the same editor for all the chain of the process, thinking that these guys know the specificity of their product better than others, but it seems that I'm not right.

    And please, stop to add a link with "More Information" who redirects to a Page Not Found.  It's very frustrating when you have a problem.  This link never sends me to a existing page regardless the kind of problem.

    Thanks

    David

    Tuesday, April 15, 2014 7:44 AM
  • Hi,

    I am also experiencing the same issue with backing up a deduped volume. The data deduplication volume backs up to disk without error but then fails when being backed up to tape with the same Error Details (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)).  

    Craig

    Tuesday, May 20, 2014 1:31 PM
  • Hello All

    Same circumstances and same errors, backing up a volume which we have recently enabled deduplication.

    Anyone managed to find a cause/solution?

    Friday, May 23, 2014 4:20 AM
  • we have the same issue - opened up a support case by microsoft


    Bueschu

    Friday, May 23, 2014 6:40 AM
  • We are having the same issue

    A Windows 2012 R2 Update 1 Datacenter DPM 2012 R2 server.

    Data Deduplication is enabled on a local volume on the host.

    Its a 2 TB NAS.

    Tape drive is a Dell PowerVault TL2000 LTO 5.

    Has anybody has any sort of resolution with the cases that they have opened?

    Thursday, May 29, 2014 4:39 PM
  • nope, but a week of pain un-optimizing everything and needing to run multiple consistency checks, and now recovery volumes that are the size of the replica volumes. (seriously hoping we can shrink those one the retention range passes)

    extremely disappointed and tainted. the dedup itself worked wonderfully on our FS but DPM didn't like 2 out of the 4 volumes we enabled dedup.


    • Edited by PCAU Friday, May 30, 2014 6:15 AM
    Friday, May 30, 2014 6:12 AM
  • Hi all,

    After the last problem with deduplicated volume for synchronisation (Microsoft take 3 month to resolve it) and now a new one to do the backup to tape we have moved to Veeam and all is working well now.

    Sorry Microsoft, I'm a supporter of your products, but let us with problem to do a backup since november (first problem, and now tape problem) is not acceptable.  You must understand that backup is critical and be more responsive on this kind of problem.

    David

    Friday, May 30, 2014 6:39 AM
  • We are also experiencing this problem on DPM 2012 R2 (RU2) on Server 2012 R2 Update 1 backing up a deduplicated volume on a Server 2012 R2 U1 file server.

    Tape drive is a Quantum Superloader3 with an LTO6 drive.

    Thursday, June 5, 2014 7:08 PM
  • We are also experiencing the same issue with 2 deduped volumes but the rest are fine.

    DPM 2012R2 RU2 Server 2012R2 Update 1

    Friday, June 13, 2014 9:26 AM
  • I finally found a work around for my environment.  Add DWORD entries with names "ConnectionNoActivityTimeout" and "ConnectionNoActivityTimeoutForNonCCJobs" in path "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent" and set the value to 7200 on DPM server.  Now recycle the DPMRA service.  Worked like a charm.

    Now, you may be wondering, why did I set the timeout to such a large value?

    Well, it was heavy handed, I tried 900 first, and the problem remained.

    At 7200 it went away.  A lower value may work just fine in your environments.


    • Edited by commoner31 Saturday, June 21, 2014 3:18 PM
    Saturday, June 21, 2014 3:18 PM
  • Microsoft has confirmed that there is an issue and they will work out a solution for a coming up DPM 2012 R2 Update Rollup

    Bueschu


    Bueschu

    • Proposed as answer by Bueschu Wednesday, June 25, 2014 4:31 AM
    Tuesday, June 24, 2014 8:40 AM
  • It dit not help me.


    Bueschu

    Tuesday, June 24, 2014 8:41 AM
  • Guys any update on this issue?

    We turned on data deduplication on our production file server last week and now backup to tape job for one of the volumes fails with

    ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)                                 

    DPM 2012R2 RU2

    Tape Library Dell TL4000 with 2 x LTO6 drives.

    Regards, Andrew

    Thursday, July 3, 2014 4:08 AM
  • Also Have the same issue on DPM 2012 R2 RU2 with File server with Dedup enabled.

    Please assist

    Thursday, August 14, 2014 8:04 AM
  • still wating for the update from ms


    Bueschu

    Friday, August 15, 2014 11:03 AM
  • We are also experiencing the same issue when we enabled deduplication on our file server. Currently running DPM 2012R2 UR3
    Monday, August 25, 2014 1:07 AM
  • Same here. I tried setting the timeouts as propopsed and even higher values, no help.
    Wednesday, September 3, 2014 9:17 AM
  • Hi,

    For the outlined issue (Tape backup jobs fail with error ID 2019 (0x80072746) I would suggest first ensure we are in supported configuration based on the below matrix. If so then the next best course of action would be to open a support case for further investigation.

    Below table shows supported / unsupported configurations for dedup protection.

     

    Protected   Server OS

    DPM   Server OS

    DPM   Versions

    Dedup   protection Supported

     

     

     

     

    Windows   2012

    Windows   2012

    DPM   2012 SP1 / R2

    Y

    Windows   2012

    Windows   2012 R2

    DPM   2012 R2

    Y

    Windows   2012 R2

    Windows   2012 R2

    DPM   2012 R2

    Y

    Windows   2012 R2

    Windows   2012

    DPM   2012 SP1/R2

    N

    Windows   2012 R2

    Windows   2012 R2

    DPM   2012 SP1

    N

      

    ***************

    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Dwayne Jackson II. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights."

    Thursday, September 4, 2014 7:13 PM
  • Unfortunately support is useless.

    We had an open case and below is the "case closure email"

    We ended up switching from volume protection to folder protection which means DPM stores its replica in de-duped state.

    Hi Andrew,

    It was my pleasure to assist you during the troubleshooting of your DPM issue. This refers to Microsoft support case ID (114070811599411).  I am providing you with a summary of the key points of the case for your records. If you have any questions please feel free to call me. You can reach me using the contact information below and referencing the case ID.

     

    ISSUE: For  F\: drive on file server ADL-SRV1 tape backup fails with error" ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)"

     

    TROUBLESHOOTING: Checked for volume corruption. 

    CAUSE: Known issue with DPM 2012 R2 - tape backup of - de-dup volumes (Bug).
     

    SOLUTION:  work around:

    Run the windows power shell command Start-DedupJob X: –Type Scrubbing –full –ReadOnly   Where X: is drive letter of deduped volume.

     

    For DPM replica – Perform these steps so that some windows power-shell commands can be ran to detect corruption:

     

    IMPORTANT NOTE: DO NOT LEAVE DEDUP ENABLED ON DPM REPLICA VOLUME.

     As per our last discussion, I am closing this support case (114070811599411). The case can be reopened anytime if you have further queries by just dropping an email to me.

    I must say that it was a pleasure to work with you.

     

    Thank you for partnering with Microsoft.

    Friday, September 5, 2014 12:03 AM
  • Thank you for sharing that naydonov.

    Any acknowledgement from MS on this "known bug" and/or upcoming resolution, i.e potentially in UR4?

    Sunday, October 12, 2014 11:38 PM
  • I can confirm that our supported configuration:

    Protected Server OS: Windows 2012 R2
    DPM Server OS: Windows 2012 R2
    DPM Server Product: DPM 2012 R2
    Supported Configuration: Y

    exhibits this problem when attempting to tape a disk replica containing a dedupicated volume.

    In light of the useless support (see case 114070811599411 above), logging this with MS is pointless.

    Our best bet is to pray that this can be fixed in an upcoming rollup.
    time to pull out the prayer mat.


    • Edited by The Fellenator Sunday, October 12, 2014 11:53 PM does eveyone have their prayer mat?
    Sunday, October 12, 2014 11:52 PM
  • Since some of these issues aren't directly the fault of DPM itself, it's possible the solution may come in the form of an update rollup for the operating system instead.

    Can anyone confirm whether the October 2014 update rollup for Windows Server 2012 R2 KB2995388, containing included fix KB2959146 for stability in Data Deduplication for Windows Server 2012 R2 fixes this problem as well?

    Thursday, October 16, 2014 10:10 AM
  • My very first test run with KB2995388 installed on the DPM server when taping a replica of a de-duped volume was successful.

    Now to see if this success continues during an overnight taping session of same.

    Tuesday, October 21, 2014 6:43 AM
  • Can you confirm your success? because we've got the same issues with tape backups of dedupped data...

    I already forwarded your findings to our case at Microsoft, just to see what Microsoft have to say about this..

    Wednesday, October 22, 2014 6:28 AM
  • I'll have to tell you on Monday, this de-duped volume only gets taped once a week on weekends.
    Wednesday, October 22, 2014 6:43 AM
  • Okay, please let me know, thanks in advance !
    Wednesday, October 22, 2014 7:46 AM
  • from my memory of this issue the first few tape backups after a successful replica creation of a deduped volume worked fine. This issue didn't appear until scheduled dedup optimization jobs ran over the course of a few days as the data changed.

    Thanks for reporting back The Fellenator, very keen to hear how it progresses although I wish MS would officially acknowledges this issue.

    Thursday, October 23, 2014 4:26 AM
  • unfortunately, it' still failing to backup to tape, so no change even with KB2995388 installed. :(
    Monday, October 27, 2014 12:22 AM
  • thank you for the tests


    Bueschu

    Monday, October 27, 2014 3:54 AM
  • Okay 2 bad, thanks for sharing the test results ..
    Monday, October 27, 2014 7:00 AM
  • Has anyone been able to test whether DPM 2012 R2 CU4 resolves this problem?

    The release notes for KB3009516 don't mention anything tape or de-dupe related as having been fixed, so I'm not hopeful.

    Maybe I should have more faith, or at least, point my prayer mat more towards Redmond?

    Tuesday, November 4, 2014 11:59 PM
  • CU4 hasn't touched this either.
    Monday, November 24, 2014 2:42 PM
  • Has anybody received an update from Microsoft about this issue, by chance? We've been unable to back up our deduplicated volumes for quite some time. Unfortunately removing deduplication is not an option and the workarounds don't work.
    Monday, December 15, 2014 5:08 PM
  • Hi @all,

    originally, we had one file server with all data on it (deduplicated volume). Then all of a sudden, the error appeared.

    After testing several tips without any effort we have moved some data to a new server with a deduplicated volume. Now backup runs smoothly on both Servers without any further errors. Seems as if the amount of data is also crucial for this problem.


    Gruß

    Ben

    MCITP Windows 7

    Wenn Dir meine Antwort hilft, markiere sie bitte entsprechend als Antwort! Danke! :-)

    Hinweis: Meine Posts werden "wie besehen" ohne jedwede Gewähr bereitgestellt, da menschliche, technische und andere Fehler nicht ausgeschlossen werden können.

    Thursday, January 8, 2015 9:00 AM
  • I noticed that our latest tape backup went through without failure, but having checked, the data has written over without the de-deduplication savings. So a volume that is on the disk backup as 500GB is going to tape as 700GB+

    Very strange.

    Friday, January 9, 2015 8:46 AM
  • Description of Update Rollup 5 for System Center 2012 R2 Data Protection Manager

    https://support.microsoft.com/kb/3021791

    Backup job for dedupe-enabled volume on Windows 2012 R2 fails. For example, Data Protection Manager backup might fail for following setup:
    • Data Protection Manager2012 R2 is installed on Windows Server 2012. 
    • Data Protection Manager is protecting a dedupe-enabled volume on Windows Server 2012 R2.

    Wednesday, February 11, 2015 1:57 AM
  • Still having the same problem, also with the DPM 2102 R2 Rollup 5 installed. protected and DPM server are all 2012R2. I am the only one left where it does not work? 

    Saturday, March 7, 2015 9:17 PM
  • Nope you are not...

    Installed the update, rebooted the server, deployed the agent updates, rebooted all of those too... For nothing... Thanks for keeping us occupied MS... hope we get a real solution soon.

    Back to surfing and googling for alternative solutions for me... Glad I've got my backup to disk, hope the building never burns down...

    Thursday, March 12, 2015 12:25 PM
  • Hey MS,

    We are getting Similar issue with the below error message:

    (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))

    Log Name:      DPM Backup Events

    Source:        DPM-BackupEvent

    Date:          13/03/2015 15:43:27

    Event ID:      33223

    Task Category: None

    Level:         Warning

    Keywords:      Classic

    User:          N/A

    Computer:     KP1.********.com

    Description:

    Backup job for datasource: F:\ on production server: f*******com failed.

     Backup job failed at: 13/03/2015 15:43:27.

     Backup Type: tape backup.

    Event Xml:

    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

      <System>

        <Provider Name="DPM-BackupEvent" />

        <EventID Qualifiers="0">33223</EventID>

        <Level>3</Level>

        <Task>0</Task>

        <Keywords>0x80000000000000</Keywords>

        <TimeCreated SystemTime="2015-03-13T15:43:27.000000000Z" />

        <EventRecordID>588</EventRecordID>

        <Channel>DPM Backup Events</Channel>

        <Computer>.*******.com</Computer>

        <Security />

      </System>

      <EventData>

        <Data>Backup job for datasource: F:\ on production server: *******.com failed.

     Backup job failed at: 13/03/2015 15:43:27.

     Backup Type: tape backup.</Data>

        <Data>33223</Data>

        <Data>488bf7fa-1c7c-4343-b5f5-38edc67f1faa</Data>

        <Data>00000000-0000-0000-0000-000000000000</Data>

        <Data>d9ba874f-b0f2-426f-8161-0fb851780e7d</Data>

        <Data>00000000-0000-0000-0000-000000000000</Data>

        <Data>2015-03-13T15:43:27</Data>

        <Data>failed</Data>

        <Data>tape</Data>

        <Data>2036</Data>

      </EventData>

    </Event>

    We also notice that MSDPMservice is crashing and below are the logs at the time of failure:

    0888 08A8 03/13 16:18:23.154 09 MsdpmService.cs(237) NORMAL OnStartAsync - Initializing MSDPM
    0888 08A8 03/13 16:18:23.233 09 servicemodule.cpp(426) NORMAL CServiceModule::InitializeSecurity
    0888 08A8 03/13 16:18:23.373 09 ApplicationSettings_expanded.cs(3790) WARNING Failed to detect DC, treating as non dc - retVal:2114
    0888 08A8 03/13 16:18:23.389 09 servicemodule.cpp(592) WARNING InitializeSecurity: Error in obtaining Account Sid for Sql Account (null).
    0888 08A8 03/13 16:18:23.545 68 DpmThreadPool.cs(121) WARNING Caught unhandled exception : System.Configuration.ConfigurationErrorsException: Unable to detect if machine is dc or not
    0888 08A8 03/13 16:18:23.545 68 DpmThreadPool.cs(121) WARNING   at Microsoft.Internal.EnterpriseStorage.Dls.Utils.DcDetector.CheckIfMachineIsDc()
    0888 08A8 03/13 16:18:23.545 68 DpmThreadPool.cs(121) WARNING   at Microsoft.Internal.EnterpriseStorage.Dls.Utils.ApplicationSettings.CheckIfMachineRoleDc()
    0888 08A8 03/13 16:18:23.545 68 DpmThreadPool.cs(121) WARNING   at Microsoft.Internal.EnterpriseStorage.Dls.Utils.ApplicationSettings.GetMSDPMTrustedMachinesAccountName()
    0888 08A8 03/13 16:18:23.545 68 DpmThreadPool.cs(121) WARNING   at Microsoft.Internal.EnterpriseStorage.Dls.Engine.CServiceModule.InitializeSecurity(CServiceModule* )
    0888 08A8 03/13 16:18:23.561 68 DpmThreadPool.cs(149) WARNING Attempting to write error to the event log Unable to detect if machine is dc or not
    0888 08A8 03/13 16:18:23.686 02 EventManager.cs(143) NORMAL Logging event to NT log from DpmThreadPool.cs(163): FatalServiceError
    0888 08A8 03/13 16:18:23.858 68 DpmThreadPool.cs(170) WARNING Attempting to invoke Watson
    0888 08A8 03/13 16:18:23.873 09 everettexception.cpp(761) CRITICAL Exception Message = Unable to detect if machine is dc or not of type System.Configuration.ConfigurationErrorsException, process will terminate after generating dump

    It seems like its a common issue with all the users around the glob without any resolution. Any formal update on this would be appreciated. 

    Saturday, March 14, 2015 2:39 PM
  • We also are having this issue as of last weekend. 
    Wednesday, March 25, 2015 1:24 PM
  • bump.
    Tuesday, March 31, 2015 1:01 PM
  • Hi,

    I have talked to MSFT in the context of another error (console crashes after RU5 installation, none of the repair advices worked) and he told me that this is still a known bug but he cannot say if there will be a hotfix or if we have to use the workaround given by nardonov.

    He told me that it could help to deactivate deduplication, create a full backup without dedup and then reactivate dedup - after RU5.

    This is of course no easy workaround. I don't know if our server can handle data without dedup or if I have to extend the volume...

    In the meantime I will try the PowerShell command.


    Gruß

    Ben

    MCSA Windows 8 (.1) MCSA Windows Server 2012 (R2)

    Wenn Dir meine Antwort hilft, markiere sie bitte entsprechend als Antwort! Danke! :-)

    Hinweis: Meine Posts werden "wie besehen" ohne jedwede Gewähr bereitgestellt, da menschliche, technische und andere Fehler nicht ausgeschlossen werden können.

    Wednesday, April 22, 2015 12:23 PM
  • Has anybody been able to confirm whether UR6 for DPM 2012 R2 resolves this?

    https://support.microsoft.com/en-us/kb/3030574

    Again I'm hopeful, since improved deduplication support is one of the mentioned fixes....

    Tuesday, May 12, 2015 7:51 AM
  • File:  Win 2012R2, Dedup Enabled

    DPM: Win 2012R2, DPM2012R2 with RU6

    It is still the same problem with RU6: (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))

    Could anybody explain what I have to do here?

    'For DPM replica – Perform these steps so that some windows power-shell commands can be ran to detect corruption:

    IMPORTANT NOTE: DO NOT LEAVE DEDUP ENABLED ON DPM REPLICA VOLUME.'


    • Edited by Bernie137 Tuesday, May 19, 2015 4:56 AM
    Monday, May 18, 2015 12:25 PM
  • File:  Win 2012R2, Dedup Enabled

    DPM: Win 2012R2, DPM2012R2 with RU6

    It is still the same problem with RU6: (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))

    Could anybody explain what I have to do here?

    'For DPM replica – Perform these steps so that some windows power-shell commands can be ran to detect corruption:

    IMPORTANT NOTE: DO NOT LEAVE DEDUP ENABLED ON DPM REPLICA VOLUME.'


    yes, the workaround suggested involves checking (and repairing) the integrity of the dedupe chunkstore of the REPLICA.  Once this has been completed, the important note is to ensure no actual de-dupe passes are run on the replica itself by the DPM server on an ongoing basis, so the idea is to ensure all de-dupe related tasks for the volume are disabled.  You DONT want it to run de-dupe real-time, and you DONT want it to run a scheduled de-dupe pass either.  In my opinion whats happening is that we're basically running a fancy version of chkdsk /f on the volume involved to repair logical corruption.  Immediately after which you should run a DPM consistency check of the volume against the protected server's volume to ensure the contents is made to re-match exactly afterwards.
    Thursday, May 28, 2015 8:04 AM
  • Even with Win2012R2, DPM2012R2 with UR6, and the protected server also being Win2012R2 and all fully patched, we're still getting the same error about an hour or so in (155GB) when taping the replica (750GB):
    "(ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))"

    Monday, June 1, 2015 7:58 AM
  • I also have the exact same problem, Fully updated, works fine on all servers apart from the deduped one
    Monday, June 1, 2015 9:00 AM
  • We're also continuing to get the same issue on the same operating systems and DPM patching levels as you.  It seems that I can occasionally get it to work if I do a consistency check and then re-run the tape job but if it does work, it will be only once before it goes back to the errors again.


    Wednesday, June 3, 2015 8:08 PM
  • Hi,

    for us, the following PowerShell command did the trick:

    Start-DedupJob -Volume <Drive letter>: -Type Scrubbing

    After that, DPM is able to fully backup all data to tape again. I cannot say whether UR6 does the same thing or makes some corrections concerning this error.


    Gruß

    Ben

    MCSA Windows 8 (.1) MCSA Windows Server 2012 (R2)

    Wenn Dir meine Antwort hilft, markiere sie bitte entsprechend als Antwort! Danke! :-)

    Hinweis: Meine Posts werden "wie besehen" ohne jedwede Gewähr bereitgestellt, da menschliche, technische und andere Fehler nicht ausgeschlossen werden können.

    Wednesday, June 10, 2015 3:04 PM
  • Hi,

    Sorry, the trick or workaround does not help for us.

    Start-DedupJob -Volume <Drive letter>: -Type Scrubbing

    Shows on our server:              

    Type               ScheduleType       StartTime              Progress   State                  Volume                       
    ----               ------------       ---------              --------   -----                  ------                       
    Scrubbing          Manual                                    0 %        Queued                 D:                           

    But backup on tape goes in the known Error 3311.

    After starting backup on tape, I see in the DPM Queue the tape backup job and waits 5 minutes but 0 MB transfered. After 5 minutes get error 3311. In the event log of file server find: 8:15:04 service DPMRA is running; 8:20:43 service DPMRA is stopped -thats all.


    • Edited by HB DBJW Thursday, June 11, 2015 6:45 AM vergessen
    Thursday, June 11, 2015 6:44 AM
  • You have to wait for the job to complete if you did not already do that ;-)

    Gruß

    Ben

    MCSA Windows 8 (.1) MCSA Windows Server 2012 (R2)

    Wenn Dir meine Antwort hilft, markiere sie bitte entsprechend als Antwort! Danke! :-)

    Hinweis: Meine Posts werden "wie besehen" ohne jedwede Gewähr bereitgestellt, da menschliche, technische und andere Fehler nicht ausgeschlossen werden können.

    Thursday, June 11, 2015 11:28 AM
  • did you run this on the DPM server?
    Monday, June 15, 2015 11:28 AM
  • No, on the protected server.

    What seems strange to me is that backing up the protected server to disk works without any problems, only backing up to tape causes this issue. What is the difference here? Does data have to be "unduplicated" before writing it to tape?


    Gruß

    Ben

    MCSA Windows 8 (.1) MCSA Windows Server 2012 (R2)

    Wenn Dir meine Antwort hilft, markiere sie bitte entsprechend als Antwort! Danke! :-)

    Hinweis: Meine Posts werden "wie besehen" ohne jedwede Gewähr bereitgestellt, da menschliche, technische und andere Fehler nicht ausgeschlossen werden können.

    Tuesday, June 16, 2015 9:05 AM
  • You have to wait for the job to complete if you did not already do that ;-)

    The job takes only 10 Seconds and is truly done on protected server before take backup starts.

    I think data have to be "unduplicated" before writing it to tape. But "unduplicating" does never start for tape backup.
    Monday, June 22, 2015 12:18 PM
  • I am definitely still having this error in my environment. I haven't been able to fully test the "fixes" in UR5 because it broke deduplication on my backups for a FS cluster (which was an exceptionally painful support ticket). I was finally able to install UR6, but the tape backup issue persists.

    Monday, June 22, 2015 3:03 PM
  • I have found a workaround for our environment.

    We have a protection group which includes 4 servers for file backup. Two of these servers are 21012r2 with dedup. Only one of them has this problem which i have discriped above. It is his volume D:\

    First i have tested the volume C:\ form the same problematic server and tape backup was no problem. Volume C:\ has no dedup enabled. Now I have removed this problematic server volume D:\ complete from the protection group and keeped all data. Next i have changed the protection group again an choose only all first level subfolders from the volume D:\ but not the root D:\ itself. Now i see all subfolders from volume D:\ in the protection group and backup on tape is NO PROBLEM! In an first test i have choosen only one subfolder of volume D:\

    Maybe is there a problem with the folder "System Volume Information" which now is not included?

    Could anyone do a test like me and reporting here?




    • Edited by HB DBJW Thursday, July 16, 2015 9:36 AM
    Thursday, July 16, 2015 9:20 AM
  • HB - I've seen this workaround mentioned before (and have seen it as a "solution" for Microsoft support cases on the issue). The problem with this is that since you're no longer backing up a volume, the backup isn't deduplicated. While this may work for environments with extra space to allocate toward backups, it most certainly will not in mine. I've had long-fought battles with MS already regarding backups that were being taken as non-optimized after the functionality was broken in UR5.

    For what it's worth, the scrubbing dedup job on the source data did NOT fix this issue in my environment.

    Thursday, July 16, 2015 2:25 PM
  • The problem with this is that since you're no longer backing up a volume, the backup isn't deduplicated.


    I think a backup on tape could never be deduplicated all files are wriiten nativ on tape and thats the reason why backup on tape is failing. While starting backup for a dedup volume anything takes too long time for "undedup" and reaches a timeout after round 5 minutes. In different environments is the number of files involved by dedup different. So in some cases backup works fine and other scenarios backup failed.
    Friday, July 17, 2015 9:29 AM
  • I had the same Problem. After installing DPM 2012 R2 UR6 on the DPM and file server and KB2995388 on the file server, I removed the datasource from the DPM Protection Group, started the scrubbing Job on the file Server and readded the Datasource. Now the tape backup works properly.
    Tuesday, July 28, 2015 7:13 AM
  • I have tried the same in the past and found that the tape jobs may work for a while but they eventually experience the same issues again.

    Maybe UR7 will help, though I see no mention of any similar issue in the list of fixes.

    Friday, July 31, 2015 5:40 PM
  • It appears that UR8 for DPM 2012 R2 might be the resolution to this issue.

    We've added a de-duped volume to a PG, it's performed several sync-to-disks on the weekdays, and last weekend it taped just fine as well.

    Hoping for a successful tape this weekend too, will advise results.

    Thursday, December 17, 2015 2:28 AM
  • the error has come back.

    after a fresh consistency check, then sync, then snapshot, taping fails after 150GB (of 1.7TB) or so with:

    (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))

    Thursday, January 21, 2016 3:02 AM
  • I still haven't been able to have a successful tape backup occur. My latest attempt occurred using a brand new DPM server (new installation of DPM on new hardware), different tape drive (upgrading from LTO-4 to LTO-5), and a different data source. After 6 hours, and almost 1.4 TB written, my latest job failed again with ID 2019.

    I did run a similar job against a much smaller data source (on a 2012 R2 server with deduplication enabled) and it was able to complete successfully. In this instance about 900 GB was backed up over the course of a few hours.

    Friday, January 29, 2016 8:06 PM
  • Any new status about this behaviour? UR 9 installed but the Problem persist...i can't belive it, Microsoft...
    Tuesday, April 12, 2016 1:56 PM
  • I can also confirm that this issue STILL EXISTS in DPM 2016 Technical Preview 4 and 5. Cannot believe this is still not fixed. We are running backups to disk and to HP 4500 Virtual Tape Library (via Fibre Channel). More often than not, backups to tape (VTL) fail.
    Tuesday, May 10, 2016 6:32 AM
  • I have disabled Dedup on my Fileservers (2012 R2). After this the Tape Backup was successfully. Has anyone test it with Windows Server 2016 TP (Fileserver + Dedup) and DPM 2016?

    • Edited by Marcel_H Tuesday, July 26, 2016 9:09 AM
    Tuesday, July 26, 2016 9:08 AM
  • Hi all,
    in general the tape backup of dedup'ed workloads should work on 2012r2 and 2016. I think the main part of the problem is that dedup show's you corruptions which are not cause by dedup but made visible through it.

    My recommendation to fix this is to run the following powershell command on the protected server (scrubbing jobs validate the integrity of all data on the volume):

    Start-DedupJob X: -Type Scrubbing -full (x is the affected volume)

    If the error still occures try to find the corrupted files in DPMRA*.errorlog (on the DPM Server) if you're not able to open them try to fix/move or delete these files.

    After fixing/deleting or excluding them from backup run a new recoverypoint job using:

    $pg = Get-DPMProtectionGroup -DPMServerName "dpmservername"

    # exec $pg to see the number (starting with 0)

    $ds = Get-DPMDatasource -ProtectionGroup $pg[X]  # (where X is you selection from the output of $pg)

    # exec $ds to see the number you need (starting with 0)

    $recoverypt = New-DPMRecoveryPoint $ds[y] -Disk -DiskrecoveryPointOption WithSynchronize -WithDataIntegrityCheck


    Tuesday, September 6, 2016 1:50 PM
  • Hello,

    FYI: I'm having this issue currently in my environment. Extremely similar setup to the original poster. I will be performing your work around this weekend and will report status sometime next week. 

    Friday, September 30, 2016 8:30 PM
  • Hi Wes8675309,

    Are you still having the same problem?  Any update that works?  I am currently having the issue with by DPM 2012 backup.


    Monday, November 21, 2016 6:49 AM
  • Hi

    I have been struggling with the same problem ever since i enabled data deduplication on our file server.

    Setup:
    DPM Host: Windows Server 2012 R2 with DPM 2012 R2 4.2.1553.0 (update rollup 11)
    Protected server: Windows Server 2012 R2, with 3 volumes protected (not c:)

    Both servers are Virtual Machines running on HyperV host with Windows Server 2012 R2, and connected to a IBM V3700 SAN

    Have tried:

    • Setting "ConnectionNoActivityTimeout" and "ConnectionNoActivityTimeoutForNonCCJobs" in path "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent"
    • Removing the Protected Volume from the Protection group, Delete all recoverypoints on disk and tape, and re-adding the Volume to the protection group.
    • Checked that tcp chimney is disabled for both the Host and Protected server.
    • Running Start-DedupJob -Volume <Drive letter>: -Type Scrubbing on the protected server.


    And still have the problem.

    But: i just tried (again) removing the protected Volume from the protection group, + deleting all restorepoints, and re-adding the Volume to the protection group. Then immediately after the new drive was added, and had created its recovery point, i tried creating a revocery point to tape. And it seems to work, as the backup has run much longer than it used to.  (used to fail at 13GB and is currently at 43GB)

    So i think the recovery point needs to be "brand new" for a tape backup to work. 

    When this test is done, I will test with another drive that is also failing, to first create a new recovery point to disk, before i create a recoverypoint to tape.  to see if that works.


    Update1: The first backup completed with no errors, and transfered all 300GB to tape. Now on to test volume 2 and 3 with a first step of recovery point to disk, before doing it to tape.

    Update2: Volume 2 complete the backup to tape, but it was also recently removed and re-added to the protection group. 
    Volume 3 failed the backup to tape,  so i will try removing that from Protection group + deleting all recovery points. and re-adding it to the protection group (again)

    Update3: After removing + deleting all recovery points for Volume 3, and re-adding the Volume to the protection Group. the backup almost completed (ran out of Tape space). So i'm currently doing a Recoverypoint to another Tape (LTO5) for all 3 Volumes. 

    Update4: The new Recovery point to tape, again fails for All volumes.  so no solution found.


    • Edited by ACSP-LBO Wednesday, December 14, 2016 9:45 AM
    Monday, December 12, 2016 10:16 AM
  • I am also Experiencing this issue when backing up to tape.

    About 12 months ago I had this issue on our DPM Server for the first time.

    Our Backup Server was DPM 2012R2 Server which was running on server 2012R2 running the latest patches which i think was about update 8/9 at the time. I tried scrubbing jobs and consistency checks but the tape backups just kept on failing.

    Pretty quickly i resolved the issue by removing the volumes from the protection groups, deleting the replicas, and re-protecting the data by only selecting folders and not the volumes themselves. This works because new replicas are created but the data is not backed up de-duped.

    Over a year later we have bought a new backup server. I have installed DPM 2016 on a Windows 2016 Server. I thought that i would try backing up the data de-duped again. After all its been so long, surely it's been fixed? 2 weeks later and I am experiencing the same issues with 5 out of 11 volumes.

    I read Heyko's comments with interest and tried scrubbing and backing up using the powershell command to check data integrity. This does not work for me. Although errors are found in the replica, my tape backups still fail on all 5 volumes.

    I would point out that I have examined the dedup logs of the protected server and no errors are ever found in the protected data during scrubbing. It's also worth mentioning that the drives are scrubbed once a week, which is i believe default behavior. These drives have been in service for 2.5 years and we have never had a single corruption issue. It's all on the DPM end in my opinion.

    I have given up again and am currently recreating the replicas without deduplication.

    I noticed that the defragger is now turned on for new drives by default. I wonder if anyone knows if this has any effect. I turned it off and did a sync with an integrity check but still had issues.

    Quite surprised that this was posted in March 2014 and we still haven't been given a solution, only a workaround.

    Tuesday, January 10, 2017 7:27 PM
  • Hard to believe it was as long ago as May 2014 when I first posted above in this thread as also having the issue. I have just moved my backup server to Windows 2016 and DPM 2016 and as the previous poster did thought I would try it again but with no success. The issue still exists!
    Saturday, January 14, 2017 10:33 AM
  • i have implemented the workaround to select folders instead of volumes, and it works.

    But now, it takes forever to backup to tape. Before i could run a backup job of about 1 tb in less than 6 hours. it now takes about double that time. (6 hours 30min for 611gb). 

    is there a simple fix for this?

    Friday, March 3, 2017 8:54 AM
  • We had a similar error - see below

    (ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746))

    This caused a dpm 2012 sp1 tape backup of a non deduped volume (W2012) to fail after 13 hrs

    We think its because the admin enabled snapshots on the volume a few minutes before the tape job failed

    The change in configuration is what did it

    Thursday, June 29, 2017 1:30 PM
  • Same trouble with relatively small volume - 90GB.

    File servers:

    Failover Cluster on Server 2012R2 with all latest updates.

    Clustered File Server role for general purpose.

    DPM 2016 Update Rollup 4 physical Server with MSL 2040 Tape library.

    Have tried:

    • Setting "ConnectionNoActivityTimeout" and "ConnectionNoActivityTimeoutForNonCCJobs" in path "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent"
    • Removing the Protected Volume from the Protection group, Delete all recoverypoints on disk and tape, and re-adding the Volume to the protection group.
    • Checked that tcp chimney is disabled for both the Host and Protected server.
    • Running Start-DedupJob -Volume <Drive letter>: -Type Scrubbing on the protected server.

    Still not fixed. I'm going to open MS Case.

    Wednesday, April 4, 2018 12:37 PM
  • Hi,

    same for us:

    DPM 2016 UR5 on Windows Server 2016 backing up a deduped vol on a Windows Server 2012.

    regards

    /bkpfast


    My postings are provided &quot;AS IS&quot; with no warranties and confer no rights

    Monday, April 8, 2019 3:02 PM