none
DPM 2016 Update Rollup 4 - Slow backups RRS feed

  • Question

  • Is anyone noticing increasingly slow backups since implementing this? My logs are showing where VHD backups would take 30 minutes before they are now running for 1.5 hours. Also the number of simulataneous backups seems to be very limited now. Many times I log in and only 1 backup is transferring data while the other jobs are queued. I have changed the registry key 

    "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\2.0\Configuration\MaxAllowedParallelBackups

    to allow more simultaneous backups but again they are still slower overall.

    Thursday, December 7, 2017 2:16 PM

All replies

  • Seeing same issue. 1 or 2 max jobs running at a time. Often times seeing high disk queue length on volume for no reason. Feels related to age of REFS volume. Feels related to the ReFS partition. Hope MS can chime in and tell us whats going on. 
    • Edited by H5TECHtoday Friday, December 8, 2017 2:42 AM
    Friday, December 8, 2017 2:37 AM
  • Can you look in your C:\Program Files\Microsoft System Center 2016\DPM\DPM\Temp\DPMRACurr.errlog

    Is anyone seeing this log being full of errors? On top of DPM being very slow, we are seeing random backups error out through the GUI with message: The DPM service was unable to communicate with the protection agent on DPMSERVER. (ID 52)

    The error log has the corresponding log entries:

    WARNING Failed: Hr: = [0x80070002] : Encountered Failure: : lVal : HRESULT_FROM_WIN32(dwError)


    Friday, December 8, 2017 1:52 PM
  • I am not seeing much in terms of errors in the logs.

    Exchange Full Database back up, 3 hours transferred 4gb of data. On 1gb network. Crazy

    Friday, December 8, 2017 6:56 PM
  • I'm watching the slowness in realtime this morning. The Hyper-V host no longer sees the backup running. It completed and merged, however, the DPM server still shows the back up running and there is quite a bit of disk activity. Seems as if it is trying to a merge or some over process after the backup completed and is just chugging.
    Sunday, December 10, 2017 1:53 PM
  • I noticed that as well. It is very noticeable with Hyper-V backups The server also feels kinda sluggish when the Hyper-V backups run. For example opening a RDP connection takes over 10 minutes.

    Monday, December 11, 2017 2:00 AM
  • Well over the past 24 hours, the problem evolved. It went from super slow with high read i/o and disk queue, to what seemed like the ReFS volume being corrupt. Drive (virtual drive) would timeout when a job would kick off. No data could be read or written to at that point. Found that the data could be read from after rebooting and making sure all agents were disabled now allowing new jobs to start. Spent the day rebuilding and splitting load into multiple DPM hosts with smaller data drives.... Lets hope this isn't a trend.
    Monday, December 11, 2017 2:26 AM
  • I think we need to be looking at https://forums.veeam.com/veeam-backup-replication-f2/refs-4k-horror-story-t40629-660.html

    Seems there is a known issue with the Refs.sys driver that is causing headaches for everyone. Specific registry setting in combination with the Refs.sys driver needs to be 10.0.14393.1934

    Not sure how to get that version other than contacting Microsoft. Are there any MS reps monitoring that can confirm? 

    Monday, December 11, 2017 4:14 PM
  • I was hoping that we will get updated refs with a cumulative update. For now you will probably have to contact Microsoft support.

    With the latest Cumulative update for Windows Server 2016 1607 (build 14393.1944) refs.sys is at version 10.0.14393.1770

    Friday, December 22, 2017 1:53 PM
  • I just speak with Microsoft and a new update is there now. kb4077525   This one is not on Windows Update automatic service.

    This update speedup my server by 1000%


    SM

    Tuesday, February 27, 2018 7:21 PM
  • HI all,

    i try install februaly cimulative update 2018-02. Version of refs library is 10.0.14393.2097. But 1tb copying 10 hours :(


    Falcon

    Sunday, March 18, 2018 10:53 AM