none
Anyone having real luck with DPM 2016 UR1 on Windows Server 2016??? RRS feed

  • Question

  • It seems to me the product has many issues that need resolving and is somewhat unstable. I don't know if they are issues related to MBR on Windows 2016 or something else.  I didn't have these problems in 2012R2.

    1) Hyper-V systems never become consistent without performing a consistency check.  The consistency checks run for hours and use large amounts of bandwidth. By the time they become consistent, it is time for another express full backup.  Consequently they are taking a lot of the system resources.

    2) Jobs take hours to complete that used to take only short periods of time.  There are always a large number of jobs running, some for hours with every little to no elapsed time. 

    3) Large numbers of event id 3 for the FilterManager in the event log.  Several disk 51 warnings and other NTFS and disk warnings. 

    4) Frequent console hanging and more frequent console crashes than in the past.

    DPM is running in Windows Server 2016 as a Hyper-V Server 2016 guest.

    Am I the only one seeing this?


    • Edited by simdoc Friday, December 16, 2016 2:42 PM
    Thursday, December 15, 2016 2:10 PM

All replies

  • Similar (bad) experiences here with DPM 2016.

    First, there's this issue: https://social.technet.microsoft.com/Forums/windowsserver/en-US/b521ed8a-937a-4c01-9ede-6633ae1c32ca/the-vhd-containing-the-replica-or-one-of-its-snapshots-could-not-be-mounted-or-unmounted-id?forum=dataprotectionmanager

    But also issues like you're describing. I upgraded my primary DPM 2012 server to DPM 2016 (OS is still Server 2012 R2). I'm using standard (not modern) storage on that DPM server.

    I have only one Server 2016 Hyper-V host at this point, and one VM on it. So that VM is protected with RCT. It's *constantly* running consistency checks. I thought Resilient Change Tracking was supposed to make things MORE efficient, not less. All my other non-RCT protected VMs are fine.

    Friday, December 16, 2016 5:29 PM
  • I left this one out of my original post as well:  https://social.technet.microsoft.com/Forums/en-US/03c61d67-859f-40d0-8a93-87beba7a2cd7/dpm-2016-ur1-console-crashes-when-adding-disk-volumes-to-secondary-protection?forum=dataprotectionmanager#e0145edb-d8f1-48b7-bb1f-3f04c0d1aa1c

    I abandoned trying to use MBS on the secondary and legacy on the primary.  So I tried to migrate the primary from legacy to MBS.  That made the primary virtually unusable, so I backed up my primary's database and retained the legacy storage offline in case I need to return to it in the future (if I possibly can).  I then started my primary over with a fresh install and database using strictly MBS.  But I still am experiencing the issues I mentioned in the original post.  I'm not sure the situation now is much better.

    All I can hope is UR2 is on its way out soon and it fixes these issues.  Right now, I cannot recommend this product.  It requires far too much interaction and I'm not confident I'm getting the backups I want at the desired frequency

    Friday, December 16, 2016 6:59 PM
  • I had to move DPM to another server so i thought it might be a good idea to go with DPM 2016 and Windows Server 2016. Modern Backup Storage is really nice as it saves a lot of space. I am not really seeing faster backups or Consistency checks though. The console often hangs for a couple of seconds after i do things like consistency checks. There also seems to be a bug with MBS. Sometimes my backups hang with ID 4002, something about a VHD that can't be mounted. Server reboot and consistency check usually fixes that.

    For the most part DPM2016 works, but is clear to me that it needs a couple more rollups to be as rock solid as DPM 2012R2 was. Typical Microsoft these days, everything feels a bit rushed. QA isn't what it used to be.


    Tuesday, December 27, 2016 10:04 AM
  • Are you backing up Hyper-V guest virtual machines that are using VHDX files (not pass-through disks) through the Hyper-V host?  Those are the systems that are having the main consistency issues.  And since they are relatively large, consistency checks take much time and bandwidth.  As far as I have observed, I've yet to see a Hyper-V VHDX backup become consistent without a consistency check.  What type of items are you protecting?

    Also, is your DPM server a Hyper-V guest as is mine so that I can eventually use deduplication as described here https://technet.microsoft.com/en-us/system-center-docs/dpm/deploy/deduplicate-dpm-storage?  However, I've not enabled deduplication yet because it would likely cause many more performance issues.

    I opened a case with MS on this issue and they have admitted there is Hyper-V bug present.  They called me earlier today saying we will simply have to wait until it is fixed (possibly UR2).

    To give you a better idea.  The number of failed jobs in my primary server is 6974.  The number of successful is 6941.  There are more failed jobs than successful ones.  These are issues I did not have with 2012R2.

    I can also add another very frequent failure with system state/BMR backups that I am observing requiring manual intervention for all of my Windows Server 2012 and 2012 R2 systems.  This results in about 30% of the recovery points actually being made that I desire.  The following event is logged in the Application log of the system being backed up:

    The backup operation that started at '‎2016‎-‎12‎-‎27T05:19:03.115498300Z' has failed with following error code '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.





    • Edited by simdoc Tuesday, December 27, 2016 5:55 PM
    Tuesday, December 27, 2016 1:36 PM
  • We had the same problem with BMR of a 2008R2 Server. Since we only need to backup one server and that will be gone in a couple of months, i worked around it. I am doing a backup with Windows Server Backup via Task Scheduler and pick up the files via DPM. 

    Our Hyper-V backups (cluster csv) are okay for the most part, had some problems with Ubuntu 16.04 backups, but that is probably a problem with the VSS daemons in the linux servers.

    We get the problem with backups hanging (ID 4002) mostly with our fileserver. The consistency check after the migration to modern backup storage took like 3 days.

    Tuesday, December 27, 2016 5:57 PM
  • What is the Hyper-V host OS?  Ours is Hyper-V Server 2016.

    Thanks.

    Tuesday, December 27, 2016 6:00 PM
  • 2012R2 cluster, CSV on iSCSI SAN.
    Wednesday, December 28, 2016 10:14 AM
  • May be the culprit is the Hyper-V 2016 system I am using.  It uses RCT now.
    • Edited by simdoc Wednesday, December 28, 2016 5:35 PM
    Wednesday, December 28, 2016 5:18 PM
  • Managed to fix the slow console by reserving more memory for the SQL server. We have SQL server installed on the same box. as DPM 2016. DPM 2016 eventually uses all available memory.
    Wednesday, January 18, 2017 10:25 AM
  • I also noticed it will consume it all as well eventually.  How much RAM do you have and how much is SQL dedicated?  We're running SQL on same box as well.  I have 24GB total, far more than minimum recommendation.
    Wednesday, January 18, 2017 2:43 PM
  • For 16 GB RAM i reserved 2 GB (minimum) for MS SQL.

    I just put some more RAM into our DPM server, it is now at 48 GB. Maybe this will help to solve some of the issues we have with DPM 2016.


    Thursday, January 19, 2017 5:05 PM
  • For 16 GB RAM i reserved 2 GB (minimum) for MS SQL.

    I just put some more RAM into our DPM server, it is now at 48 GB. Maybe this will help to solve some of the issues we have with DPM 2016.


    We upped ours from 16 to 32GB of ram and it didn't seem to make a difference.  The server just started throwing the same errors when at 78% ram instead of 95%.
    Friday, January 20, 2017 8:13 PM
  • At the recommendation of Microsoft, we changed the page file size to follow the article below.  Performance does seem to be better, although I'm still having issues with BMR failure.  https://technet.microsoft.com/en-us/library/hh757757(v=sc.12).aspx
    Friday, January 20, 2017 8:44 PM
  • After making the page file change and running this system for a few days, performance is minimally better, if at all.  I'm still seeing the same problems as before.
    Tuesday, January 24, 2017 5:22 PM