Data Protection Manager - General announcement
-
Link
We have seen DPM 2016 or later servers using Modern Backup Storage (MBS) experience the below error.
The VHD containing the replica or one of it's snapshots could not be mounted or unmounted. (ID 40002)
This can be caused by a real failure to mount / dismount a replica or snapshot .vhdx file – or most likely the mount / dismount is working but WMI call is failing due to WMI out of memory error.
NOTE: We have seen some customers experience this problem when using volume mountpoints for modern backup storage (MBS) so it suggested to assign a drive letter to the MBS volume(s) , then refresh the DPM Storage pool so it picks up the drive letter, then remove the mount point.
To mitigate the WMI memory problem you can implement the below changes.
1) Add these registry settings. Copy / paste into notepad and save as WMI.REG (no .txt extension) - then right-click the wmi.reg and click merge.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wbem\CIMOM]
"High Threshold On Client Objects (B)"="60000000"
"High Threshold On Events (B)"="60000000"2) Follow the below blog and change memory from 512mb to 1GB.
Memory and Handle Quotas in the WMI Provider Service
https://techcommunity.microsoft.com/t5/Ask-The-Performance-Team/Memory-and-Handle-Quotas-in-the-WMI-Provider-Service/ba-p/3733191.Start “wbemtest” as administrator.
2.Connect to the “root” namespace (not “root\default”, just “root”)
3.Click Open Instance. Specify “__ProviderHostQuotaConfiguration=@”
4.Select Local Only for easier readability. You will see the threshold values.
5.Increase the MemoryPerHost value to something greater. (For example, double it to 1GB 1073741824)
6.Save Property
7.Save Object
8.Click Exit.3) Reboot the DPM Server.
-
Link
On March 7, 2019, we shared that System Center 2019 would be coming soon. As of March 14, 2019, we are pleased to let you know that System Center 2019 is generally available. Customers with a valid license of System Center 2019 can download media from the Volume Licensing Service Center (VLSC). We will also have the System Center 2019 evaluation available on the Microsoft Evaluation Center.
Faster backups with Data Protection Manager 2019
Data Protection Manager (DPM) 2019 will provide backups optimized in time (faster) and space (consumes less storage).
- DPM improves performance of your backups with a 75 percent increase in speed and enables monitoring experience for key backup parameters via Log Analytics.
- DPM further supports backup of VMware VMs to tape. In addition to Windows Server 2019, DPM now provides backups for new workloads such as SharePoint 2019 and Exchange 2019.
-
Link
ISSUE: BMR backups fail
Issue occurs on Data Protection Manager\Microsoft Azure Backup Server 2016 using Modern Backup Storage. This occurs because windows backup server writers directly to the DPM\MABS share during backup and there could be IO latency greater than the standard SMB timeout of 45-60 seconds. There are also some general items that should be configured for BMR backups please see Pre-requisites below.
Pre-Requisites:
Latest UR patch for Data Protection Manager or Microsoft Azure Backup Server
Ensure Replica volume is sized correctly – https://blogs.technet.microsoft.com/dpm/2017/04/14/how-to-increase-dpm-2016-replica-when-using-modern-backup-storage-mbs/
Ensure July CU update or above is applied for REFS - https://support.microsoft.com/en-us/help/4345418/windows-10-update-kb4345418
Symptoms:
On DPM server:
DPM cannot create a backup because Windows Server Backup (WSB) on the protected computer encountered an error (WSB Event ID: 517, WSB Error Code:
0x6B48C3A0). (ID 30229 Details: Internal error code: 0x80990ED0)
OR
You may also encounter an error “The storage involving the current operation could not be read from or written to. (ID 40003)”
Log Name: DPM Alerts
Source: DPM-EM
Date: 7/7/2017 11:11:36 AM
Event ID: 3106
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: <servername>
Description:
The replica of Non VSS Datasource Writer on <servername> is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. (ID: 3106)
The storage involving the current operation could not be read from or written to. (ID: 40003)
On Protected server:
Log Name: Microsoft-Windows-Backup
Source: Microsoft-Windows-Backup
Date: 5/8/2018 10:43:42 AM
Event ID: 5
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: <Servername>
Description:
The backup operation that started at '2018-05-08T15:17:32.312386000Z' has failed with following error code '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
And
Log Name: Microsoft-Windows-SmbClient/Connectivity
Source: Microsoft-Windows-SMBClient
Date: 5/21/2018 4:23:49 PM
Event ID: 30809
Task Category: None
Level: Error
Keywords: (64)
User: N/A
Computer: <servername>
Description:
A request timed out because there was no response from the server.
And
Server name: <Servername>
Session ID:0x17801D0000031
Tree ID:0x1
Message ID:0x4000015
Command: Flush
Guidance:
The server is responding over TCP but not over SMB. Ensure the Server service is running and responsive, and the disks do not have high per-IO latency, which makes the disks appear unresponsive to SMB. Also, ensure the server is responsive overall and not paused; for instance, make sure you can log on to it.
Log Name: Microsoft-Windows-SMBServer/Operational
Source: Microsoft-Windows-SMBServer
Date: 5/21/2018 4:25:37 PM
Event ID: 1020
Task Category: (1020)
Level: Warning
Keywords: (8)
User: SYSTEM
Computer: <ServerName>
Description:
File system operation has taken longer than expected.
Client Name: <IP>
Client Address: <IP:Port>
User Name: <Username>
Session ID: 0x17801D0000031
Share Name: <sharename>
File Name: WINDOWSIMAGEBACKUP\<Servername>\<Backupdate>\<GUID>.VHDX
Command: 7
Duration (in milliseconds): 353151
Warning Threshold (in milliseconds): 120000
Guidance:
The underlying file system has taken too long to respond to an operation. This typically indicates a problem with the storage and not SMB
Cause:
SMB timeout is too short for IO latency for BMR on REFS
Resolution\Workaround:
Option 1:
Set registry and reboot protected server
- Add the registry key below on the protected server
reg add HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters /v SessTimeout /t REG_DWORD /d 1200
Option 2: If you cannot reboot server (Note: Ensure you check the list of dependencies for the Lanmanworkstation service as those services will need to be started again too)
- Add the registry key below on the protected server
reg add HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters /v SessTimeout /t REG_DWORD /d 1200
Note: The 1200 second setting is a 20 minute timeout. This is not a rigid number but has been tested with success. Depending on latency you can adjust as needed. To get suspected latency please review REFS event logs on DPM\MABS server.
- Stop/Start Lanmanworkstation service
Note: There are some other services that depend on Lanmanworkstation, just start those back.
-
Link
There is one optimization change you can make in DPM 2016 using Modern Backup Storage that “may” help reduce the backup times. The issue is that DPM tries to get the size of the recovery point by asking ReFs to count the blocks associated with that recovery point so it can update the DPMUI “storage used” in the UI.
To eliminate that additional calculation that can take a long time and allow the next backup to start you can disable that calculation using the below DPM powershell command.
Manage-DPMDSStorageSizeUpdate.ps1 -ManageStorageInfo StopSizeAutoUpdate
As a result, the “storage consumed” for each data source will not be displayed after a new backup is completed.
The above is covered in the release notes: https://docs.microsoft.com/en-us/system-center/dpm/dpm-release-notes
Section titled: Recovery Points not being pruned, leading to an accumulation of Recovery Points.
If you need to know the amount used for a data source you can use the associated UpdateSizeForDS command.
-
Link
There may be circumstances where the replica volume for a protected data source is under allocated or a very large increase of protected data results in synchronization or recovery point job failures due to inadequate space on the replica.
Type: Recovery point
Status: Failed
Description: DPM is out of disk space for the replica. (ID 58 Details: There is not enough space on the disk (0x80070070))As in previous versions, DPM 2016 has replica auto-grow feature for modern backup storage (MBS) to help increase the replica sizes as the data sources grows. Under normal use conditions this feature works perfectly fine. If a job fails due to out of disk space, it will automatically grow the replica volume and schedule a new synchronization or recovery point job to run one hour in the future. However, if the amount of new data is much larger than the size of the replica after auto-grow, the jobs will continue to fail even as the auto-grow continues to increase the replica size. After two failures, auto-rerun is no longer triggered by default. In DPM 2012 R2 you had the ability to manually grow the replica volume as large as you wanted by using the “modify disk allocation” wizard, however in the DPM 2016 you do not have the ability to manually grow the replica volume using the same wizard. This can cause delays in getting good backups again to meet your SLA. If you know the new size of the protected data or want to grow the replica volume for future data growth in order to prevent future backup failures, you can grow the replica manually.
The below BLOG contains a DPM PowerShell script to make it easy to manually increase a replica volume to the new desired size.
How to increase DPM 2016 replica when using Modern Backup Storage (MBS)
https://techcommunity.microsoft.com/t5/System-Center-Blog/How-to-increase-DPM-2016-replica-when-using-Modern-Backup/ba-p/351818 -
Link
When using DPM 2016 to protect SystemState / BMR workloads the storage used for those backups will be higher when using DPM Modern Backup Storage (MBS) as compared to using legacy Logical Disk Manager (LDM) based storage used in DPM 2012 R2.
When DPM initiates a SystemState or BMR backup, Windows Server Backup is used and overwrites the existing WindowsImagebackup folder with the new backup data. For SystemState that folder is on the protected server and later transferred to the DPM Server replica. For BMR, Windows Server Backup overwrites the folder on the Replica volume directly.
When the replica is located on a LDM based volume, Volume Shadow Copy (VSS) snapshots are used for recovery points. When the files are overwritten on the replica volume, VSS performs a read for the original file blocks – compares them to what is about to be written - then either copies the original blocks to the shadow copy (recovery point volume) if different or just lets the write thru if the blocks are identical. This is called Copy On Write (COW). Between backups a lot of the blocks are the same between the files so the shadow copies were relatively small.
Modern Backup Storage (MBS) uses ReFs clones for recovery points. There is no copy on write operation like there are when using VSS, all writes are performed but instead of overwriting the original files the new blocks are written to different locations on the MBS volume. This is called allocate on write, so even blocks that are the same will use different free space on the Refs volume. That results in more storage consumption as each backup will consume the amount of space as a complete SystemState or BMR backup. IE: If the initial replica size shows 24GB used, then all subsequent backups will come very close to consuming the same amount of space. If the backups are taken frequently or have a long retention period the storage usage can be significant.
To mitigate this behavior I would suggest either take fewer BMR / SystemState recovery points, use a reduced retention period, or just plan on needing more storage for SystemState / BMR backups.
-
Link
System Center Data Protection Manager 2012 R2 UR11 added support for VMware host level backup. If you attempt to perform an Item Level Recovery (ILR) from the latest recovery point for a Protected Windows VM, you receive an ID: 958 popup error message with the below details.
IMPORTANT NOTE: This problem is not present in Microsoft Azure Backup Server (MABS) update rollup 1 (UR1) that also added support for VMWare host level backups.
WORKAROUND(S)
===========1) Select an earlier recovery point if the file(s) (version) you want to recover was included in an earlier recovery point.
2) Restore the entire VM to an alternate location and copy the desired file(s) out of that restored VM.
3) Create a new AD-HOC recovery point manually. That will enable ILR to work for all recovery points including the latest one you made manually.UPDATE: This issue is now fixed in DPM 2012 R2 UR13. However, you need to consider the following for the fix to work.
1. For Protection Groups created after applying UR13, Item Level Recovery will work fine.
2. For older Protection Groups, you need to modify the PGs by adding/removing a small VM or making some configuration changes to the PG for the fix to work. - Link
-
Link
SC DPM 2016 brings in features that delivers improvements in multiple key areas of backup efficiency, performance, flexibility and security. With Modern Backup Storage (MBS), DPM 2016 is turning around the way backups are stored by leveraging modern technologies like ReFS Block Cloning, ReFS Allocate On Write and VHDX. This to 3X faster backups and 50% reduction in storage consumption thus reducing overall backup TCO.
https://blogs.technet.microsoft.com/dpm/2016/10/19/announcing-sc-dpm-2016-with-modern-backup-storage/ -
Link
DPM 2016 comes with Modern Backup Storage which delivers 50% storage savings, 3x faster backups, and more efficient storage with Workload-Aware storage. Modern Backup Storage can be used by setting up DPM 2016 on Windows Server 2016.
Add Storage to DPM 2016 -
Link
Very happy to report the release SC DPM 2012 R2 UR11 that adds protection for VMware host level backup of VMs. Here are major features of VMware VM Backup. Here are the few highlights of VMware VM backup.
- Agentless backup
- Backup at vCenter or ESXi level
- Backup VMware VM folders
- Ability to do ILR
- Protect large data center using DPM Scale Out
- Backup to Disk/Cloud
Here are the blogs and articles that are published.
BLOG - Implementing Microsoft DPM host level protection of VMware VMs
BLOG-Announcing VMware VM Backup using System Center Data Protection Manager
-
Link
We have made a bunch of improvements in this update, related to reboot requirements, Enterprise file server backup, online backups and more.
-
Link
After you install DPM 2012 R2 UR7 update (KB3065246), DPM cannot delete expired recovery points. This will lead to extra recovery point volume usage.
Please download and install the fix for this issue.
Fix for a known issue with Update Rollup 7 for System Center 2012 R2 Data Protection Manager
MORE INFORMATION
==============DPM 2012 R2 UR7 introduced a new switch to the powershell cmdlet Get-RecoveryPoint, it now requires a -Datasource switch as seen below:
Get-RecoveryPoint -Datasource $ds
If you have any custom powershell scripts that use that cmdlet, please modify them to use the new -Datasource switch.
-
Link
Below are some handy links for the DPM Console troubleshooter which can help you resolve console crashes and three DPMDB maintenance blogs.
Guided Walkthrough for Troubleshooting Data Protection Manager console issues
DPMDB Maintenance Part 1: Database consistency check and your DPMDB
DPMDB Maintenance Part 2: Identifying and dealing with fragmentation in your DPMDB
DPMDB Maintenance Part 3: Dealing with a large DPMDB -
Link
Hello - Please find the following BLOG on the new streamlined method of registering a DPM 2012 SP1 or later DPM server to backup to Azure. There is no longer a requirement to upload a certificate when registering a DPM server for online backups.
Below are two Blogs on the subject.
How to register a DPM 2012 SP1 or later server to backup to Microsoft Azure
Simplified workflow to configure backup to Azure from DPM -
Link
Please find the following whitepaper detailing how to dedup DPM storage pool to help reduce overall disk storage requirements for disk based backups. This solution requires Windows 2012 R2 and System Center Data Protection Manager 2012 R2
-
Link
For DPM Console crash issues, please review this troubleshooter before posting a new question.
Guided Walkthrough for Troubleshooting Data Protection Manager console issues
-
Link
updated 6-5-13
Hello Everyone,
System Center DPM 2012 is not supported running on Windows Server 2012. We have had reports of DPM 2012 console crashes when navigating to the recovery tab if DPM 2012 is installed on Windows server 2012. This issue is fixed in System Center 2012 SP1 which supports being installed on Windows 2012 operating system.
If you are already in this condition, you have two options:
A) You can upgrade to System Center 2012 Sp1.
B) You can backup the DPMDB, rebuild the DPM server using Windows 2008 R2 install DPM 2012, then restore your current database. The process is explained in the below technet article.
Recovering DPM servers
http://technet.microsoft.com/en-us/library/jj244606.aspxWe also do not support protecting Windows Server 2012 workloads using SCS DPM 2012, you must upgrade to DPM 2012 Sp1. Please do not expect any support for DPM 2012 running on Windows Server 2012, or issues while protecting data sources in Windows Server 2012.
-
0 VotesDPM 2016 crashes after installing UR10
I get DPM service failure when I try to add an object. Got an error on the new server, reproduced it on the old one, which I updated from UR9 to UR10. Old storage on both, not ...Unanswered | 20 Replies | 3276 Views | Created by tche - Monday, November 2, 2020 10:30 PM | Last reply by Grant Thompson - Tuesday, December 29, 2020 6:48 PM
-
0 Votes
Microsoft Azure Backup Server Console crashes - DPMAM Service crashes after MABS Upgrade to V3
Hello, I have a Microsoft Azure Backup Server configured in a clustered environment. It was on V2 and needed to upgrade, tried to upgrade to V3 following the usual upgrade procedure ...Proposed | 1 Replies | 2197 Views | Created by braynyak - Tuesday, December 8, 2020 10:32 AM | Last reply by Tome Lopes - Tuesday, December 29, 2020 6:12 PM -
0 Votes
back up
I have a notification that I need to back up windows media, in the process of doing so, I am told It must be formatted, if I format it I will completely erase all data on the ...Unanswered | 1 Replies | 856 Views | Created by theresajengle - Sunday, December 27, 2020 7:19 PM | Last reply by Leon Laude - Monday, December 28, 2020 3:27 PM -
0 Votes
Norton Multifactor Authenticator
Why did my Norton Multifactor Authentication key disappear from the Microsoft Aurhenticator?Unanswered | 2 Replies | 964 Views | Created by e_ent - Sunday, December 27, 2020 2:08 PM | Last reply by Leon Laude - Monday, December 28, 2020 3:26 PM -
0 Votes
Warnning Consolidation of recovery points of the replica
Hello, I have system center 2019 installed with a 2019 operating system. Everything was working fine until most of the protection groups started throwing the warning "Consolidation of recovery ...Unanswered | 0 Replies | 4761 Views | Created by Carlos Augusto Gonzalez - Wednesday, December 23, 2020 4:10 PM -
0 Votes
Online Recovery Point Creation Failed
Hi, My DPM server online recovery points creation failed with the below error. DPM Version : Microsoft Azure Backup Server (Version 12.0.332.0) ...Unanswered | 0 Replies | 1354 Views | Created by Gayan Sanjeewa Perera - Tuesday, December 22, 2020 10:13 AM -
0 Votes
Authenticator QR Code
I was having issues with my authenticator app on my Samsung S20 so I deleted the app and reinstalled it and now i can't get a QR code. Please help as I'm 50 and a technological retard........Proposed | 1 Replies | 1197 Views | Created by Help_Me_Please_MissN - Tuesday, December 22, 2020 7:16 AM | Last reply by Leon Laude - Tuesday, December 22, 2020 7:32 AM -
0 Votes
DPM database size has exceeded the threshold limit
Hi Everyone, could you help me to resolved this issue see below DPM database size has exceeded the threshold limit. DPM database size: 3.39 ...Proposed | 2 Replies | 1564 Views | Created by Mooner09 - Wednesday, December 16, 2020 1:17 AM | Last reply by Mike Jacquet - Thursday, December 17, 2020 8:48 PM -
1 Votes
Data co-location on tape fails
Due to a server failure, we had to reinstall our DPM server and restore the DPM DB. Everything is now up and running, but the colocation of data on tape fails. Even though a Co-located protection ...Answered | 9 Replies | 1621 Views | Created by Jaan Meijer - Friday, November 4, 2016 10:39 AM | Last reply by AndyChips - Thursday, December 17, 2020 12:58 PM -
0 Votes
(Sorry if this is not the correct forum) BitLocker for Windows10 Home?
Please direct me to information on how to obtain/enable BitLocker for Windows10 Home.Proposed | 1 Replies | 1416 Views | Created by CortanaisaSpy - Thursday, December 17, 2020 5:02 AM | Last reply by Leon Laude - Thursday, December 17, 2020 7:56 AM -
0 Votes
Required Group Policy Settings for DPM functionality on Windows 10 clients?
Gents, We have DPM setup in our environment however we cannot get it to function with a slew of various errors ranging from communication/synch, etc. because of our General ...Unanswered | 1 Replies | 1320 Views | Created by Landon V - Friday, December 11, 2020 7:32 PM | Last reply by Landon V - Sunday, December 13, 2020 6:29 AM -
0 Votes
Back and Restore tools/ways to compare backed up VM image with current VM state
Do we have any preferred tools to compare a VM to a previous back up, i.e. compare and contrast from current state to previous state? -
3 Votes
DPM 2019 UR2 - Crashes on PG change/creation
Hello, after instaling UR2 for SC DPM 2019 DPM server crashes on Protection group creation or modification - adding protected source. I try to add simple folder from already ...Discussion | 32 Replies | 4715 Views | Created by GP_WilL_ - Tuesday, August 25, 2020 10:29 AM | Last reply by Grant Thompson - Friday, December 11, 2020 6:25 PM -
0 Votes
DPM 2019 crashes almost every night with 100% cpu
Hi, I have two DPM 2019 servers with sql on another server. Backups are running without problems on both servers. But one of the servers crashes almost every night and I cannot ...Unanswered | 1 Replies | 1876 Views | Created by siliconbase - Thursday, December 10, 2020 10:55 AM | Last reply by Aayoosh Moitro - Thursday, December 10, 2020 4:58 PM -
0 Votes
How to control software in local VM
Hi all, In our company we use SCCM to control all domain computers, and apply security policies and compliances. Users also use VM (Hyper-V and Virtualbox) in ...Answered | 2 Replies | 959 Views | Created by zbroinx - Wednesday, December 9, 2020 5:44 PM | Last reply by zbroinx - Thursday, December 10, 2020 11:19 AM -
0 Votes
Bulk Windows 10 pro devices backup and restore data
Dear all, I'm currently on a project where the environment wil start using Azure hybrid AD and system manager. 500+ new windows 10 pro desktops/laptops will ... -
0 Votes
Can I recover DPM 2012 data stored in Azure from DPM 2019
I currently have a DPM 2012 server that is backing up locally to an iSCSI NAS and to an Azure vault. I would like to install a new DPM 2019 server to replace it. My NAS does not have any ...Unanswered | 1 Replies | 793 Views | Created by Ben Dalton - Wednesday, December 2, 2020 7:55 PM | Last reply by Leon Laude - Wednesday, December 2, 2020 9:08 PM -
0 Votes
DPM 2019 Slow Performance
We support an environment in which DPM 2019 UR 2 is installed on Windows Server 2019 with direct attached storage (volume of SAS disks in a RAID 5 configuration). This configuration is taking ...Unanswered | 2 Replies | 1600 Views | Created by nick_the_first - Monday, November 9, 2020 8:19 PM | Last reply by nick_the_first - Tuesday, December 1, 2020 3:35 PM -
0 Votes
When I open an app such as roblox, or spotify on my computer, why does the hand change to a fist and not allow me to click anything?
I have a samsung chromebook, used. and recently, when ever i open an app this happens; It'll load, but as soon as it does the mouse changes into a fist as if its grabbing something. I really have no ...Unanswered | 1 Replies | 2996 Views | Created by xXwinne_da_poohXx - Tuesday, December 1, 2020 12:35 PM | Last reply by Leon Laude - Tuesday, December 1, 2020 2:25 PM - Items 1 to 20 of 5562 Next ›
Data Protection Manager - General announcement
-
Link
We have seen DPM 2016 or later servers using Modern Backup Storage (MBS) experience the below error.
The VHD containing the replica or one of it's snapshots could not be mounted or unmounted. (ID 40002)
This can be caused by a real failure to mount / dismount a replica or snapshot .vhdx file – or most likely the mount / dismount is working but WMI call is failing due to WMI out of memory error.
NOTE: We have seen some customers experience this problem when using volume mountpoints for modern backup storage (MBS) so it suggested to assign a drive letter to the MBS volume(s) , then refresh the DPM Storage pool so it picks up the drive letter, then remove the mount point.
To mitigate the WMI memory problem you can implement the below changes.
1) Add these registry settings. Copy / paste into notepad and save as WMI.REG (no .txt extension) - then right-click the wmi.reg and click merge.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wbem\CIMOM]
"High Threshold On Client Objects (B)"="60000000"
"High Threshold On Events (B)"="60000000"2) Follow the below blog and change memory from 512mb to 1GB.
Memory and Handle Quotas in the WMI Provider Service
https://techcommunity.microsoft.com/t5/Ask-The-Performance-Team/Memory-and-Handle-Quotas-in-the-WMI-Provider-Service/ba-p/3733191.Start “wbemtest” as administrator.
2.Connect to the “root” namespace (not “root\default”, just “root”)
3.Click Open Instance. Specify “__ProviderHostQuotaConfiguration=@”
4.Select Local Only for easier readability. You will see the threshold values.
5.Increase the MemoryPerHost value to something greater. (For example, double it to 1GB 1073741824)
6.Save Property
7.Save Object
8.Click Exit.3) Reboot the DPM Server.
-
Link
On March 7, 2019, we shared that System Center 2019 would be coming soon. As of March 14, 2019, we are pleased to let you know that System Center 2019 is generally available. Customers with a valid license of System Center 2019 can download media from the Volume Licensing Service Center (VLSC). We will also have the System Center 2019 evaluation available on the Microsoft Evaluation Center.
Faster backups with Data Protection Manager 2019
Data Protection Manager (DPM) 2019 will provide backups optimized in time (faster) and space (consumes less storage).
- DPM improves performance of your backups with a 75 percent increase in speed and enables monitoring experience for key backup parameters via Log Analytics.
- DPM further supports backup of VMware VMs to tape. In addition to Windows Server 2019, DPM now provides backups for new workloads such as SharePoint 2019 and Exchange 2019.
-
Link
ISSUE: BMR backups fail
Issue occurs on Data Protection Manager\Microsoft Azure Backup Server 2016 using Modern Backup Storage. This occurs because windows backup server writers directly to the DPM\MABS share during backup and there could be IO latency greater than the standard SMB timeout of 45-60 seconds. There are also some general items that should be configured for BMR backups please see Pre-requisites below.
Pre-Requisites:
Latest UR patch for Data Protection Manager or Microsoft Azure Backup Server
Ensure Replica volume is sized correctly – https://blogs.technet.microsoft.com/dpm/2017/04/14/how-to-increase-dpm-2016-replica-when-using-modern-backup-storage-mbs/
Ensure July CU update or above is applied for REFS - https://support.microsoft.com/en-us/help/4345418/windows-10-update-kb4345418
Symptoms:
On DPM server:
DPM cannot create a backup because Windows Server Backup (WSB) on the protected computer encountered an error (WSB Event ID: 517, WSB Error Code:
0x6B48C3A0). (ID 30229 Details: Internal error code: 0x80990ED0)
OR
You may also encounter an error “The storage involving the current operation could not be read from or written to. (ID 40003)”
Log Name: DPM Alerts
Source: DPM-EM
Date: 7/7/2017 11:11:36 AM
Event ID: 3106
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: <servername>
Description:
The replica of Non VSS Datasource Writer on <servername> is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. (ID: 3106)
The storage involving the current operation could not be read from or written to. (ID: 40003)
On Protected server:
Log Name: Microsoft-Windows-Backup
Source: Microsoft-Windows-Backup
Date: 5/8/2018 10:43:42 AM
Event ID: 5
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: <Servername>
Description:
The backup operation that started at '2018-05-08T15:17:32.312386000Z' has failed with following error code '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
And
Log Name: Microsoft-Windows-SmbClient/Connectivity
Source: Microsoft-Windows-SMBClient
Date: 5/21/2018 4:23:49 PM
Event ID: 30809
Task Category: None
Level: Error
Keywords: (64)
User: N/A
Computer: <servername>
Description:
A request timed out because there was no response from the server.
And
Server name: <Servername>
Session ID:0x17801D0000031
Tree ID:0x1
Message ID:0x4000015
Command: Flush
Guidance:
The server is responding over TCP but not over SMB. Ensure the Server service is running and responsive, and the disks do not have high per-IO latency, which makes the disks appear unresponsive to SMB. Also, ensure the server is responsive overall and not paused; for instance, make sure you can log on to it.
Log Name: Microsoft-Windows-SMBServer/Operational
Source: Microsoft-Windows-SMBServer
Date: 5/21/2018 4:25:37 PM
Event ID: 1020
Task Category: (1020)
Level: Warning
Keywords: (8)
User: SYSTEM
Computer: <ServerName>
Description:
File system operation has taken longer than expected.
Client Name: <IP>
Client Address: <IP:Port>
User Name: <Username>
Session ID: 0x17801D0000031
Share Name: <sharename>
File Name: WINDOWSIMAGEBACKUP\<Servername>\<Backupdate>\<GUID>.VHDX
Command: 7
Duration (in milliseconds): 353151
Warning Threshold (in milliseconds): 120000
Guidance:
The underlying file system has taken too long to respond to an operation. This typically indicates a problem with the storage and not SMB
Cause:
SMB timeout is too short for IO latency for BMR on REFS
Resolution\Workaround:
Option 1:
Set registry and reboot protected server
- Add the registry key below on the protected server
reg add HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters /v SessTimeout /t REG_DWORD /d 1200
Option 2: If you cannot reboot server (Note: Ensure you check the list of dependencies for the Lanmanworkstation service as those services will need to be started again too)
- Add the registry key below on the protected server
reg add HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters /v SessTimeout /t REG_DWORD /d 1200
Note: The 1200 second setting is a 20 minute timeout. This is not a rigid number but has been tested with success. Depending on latency you can adjust as needed. To get suspected latency please review REFS event logs on DPM\MABS server.
- Stop/Start Lanmanworkstation service
Note: There are some other services that depend on Lanmanworkstation, just start those back.
-
Link
There is one optimization change you can make in DPM 2016 using Modern Backup Storage that “may” help reduce the backup times. The issue is that DPM tries to get the size of the recovery point by asking ReFs to count the blocks associated with that recovery point so it can update the DPMUI “storage used” in the UI.
To eliminate that additional calculation that can take a long time and allow the next backup to start you can disable that calculation using the below DPM powershell command.
Manage-DPMDSStorageSizeUpdate.ps1 -ManageStorageInfo StopSizeAutoUpdate
As a result, the “storage consumed” for each data source will not be displayed after a new backup is completed.
The above is covered in the release notes: https://docs.microsoft.com/en-us/system-center/dpm/dpm-release-notes
Section titled: Recovery Points not being pruned, leading to an accumulation of Recovery Points.
If you need to know the amount used for a data source you can use the associated UpdateSizeForDS command.
-
Link
There may be circumstances where the replica volume for a protected data source is under allocated or a very large increase of protected data results in synchronization or recovery point job failures due to inadequate space on the replica.
Type: Recovery point
Status: Failed
Description: DPM is out of disk space for the replica. (ID 58 Details: There is not enough space on the disk (0x80070070))As in previous versions, DPM 2016 has replica auto-grow feature for modern backup storage (MBS) to help increase the replica sizes as the data sources grows. Under normal use conditions this feature works perfectly fine. If a job fails due to out of disk space, it will automatically grow the replica volume and schedule a new synchronization or recovery point job to run one hour in the future. However, if the amount of new data is much larger than the size of the replica after auto-grow, the jobs will continue to fail even as the auto-grow continues to increase the replica size. After two failures, auto-rerun is no longer triggered by default. In DPM 2012 R2 you had the ability to manually grow the replica volume as large as you wanted by using the “modify disk allocation” wizard, however in the DPM 2016 you do not have the ability to manually grow the replica volume using the same wizard. This can cause delays in getting good backups again to meet your SLA. If you know the new size of the protected data or want to grow the replica volume for future data growth in order to prevent future backup failures, you can grow the replica manually.
The below BLOG contains a DPM PowerShell script to make it easy to manually increase a replica volume to the new desired size.
How to increase DPM 2016 replica when using Modern Backup Storage (MBS)
https://techcommunity.microsoft.com/t5/System-Center-Blog/How-to-increase-DPM-2016-replica-when-using-Modern-Backup/ba-p/351818 -
Link
When using DPM 2016 to protect SystemState / BMR workloads the storage used for those backups will be higher when using DPM Modern Backup Storage (MBS) as compared to using legacy Logical Disk Manager (LDM) based storage used in DPM 2012 R2.
When DPM initiates a SystemState or BMR backup, Windows Server Backup is used and overwrites the existing WindowsImagebackup folder with the new backup data. For SystemState that folder is on the protected server and later transferred to the DPM Server replica. For BMR, Windows Server Backup overwrites the folder on the Replica volume directly.
When the replica is located on a LDM based volume, Volume Shadow Copy (VSS) snapshots are used for recovery points. When the files are overwritten on the replica volume, VSS performs a read for the original file blocks – compares them to what is about to be written - then either copies the original blocks to the shadow copy (recovery point volume) if different or just lets the write thru if the blocks are identical. This is called Copy On Write (COW). Between backups a lot of the blocks are the same between the files so the shadow copies were relatively small.
Modern Backup Storage (MBS) uses ReFs clones for recovery points. There is no copy on write operation like there are when using VSS, all writes are performed but instead of overwriting the original files the new blocks are written to different locations on the MBS volume. This is called allocate on write, so even blocks that are the same will use different free space on the Refs volume. That results in more storage consumption as each backup will consume the amount of space as a complete SystemState or BMR backup. IE: If the initial replica size shows 24GB used, then all subsequent backups will come very close to consuming the same amount of space. If the backups are taken frequently or have a long retention period the storage usage can be significant.
To mitigate this behavior I would suggest either take fewer BMR / SystemState recovery points, use a reduced retention period, or just plan on needing more storage for SystemState / BMR backups.
-
Link
System Center Data Protection Manager 2012 R2 UR11 added support for VMware host level backup. If you attempt to perform an Item Level Recovery (ILR) from the latest recovery point for a Protected Windows VM, you receive an ID: 958 popup error message with the below details.
IMPORTANT NOTE: This problem is not present in Microsoft Azure Backup Server (MABS) update rollup 1 (UR1) that also added support for VMWare host level backups.
WORKAROUND(S)
===========1) Select an earlier recovery point if the file(s) (version) you want to recover was included in an earlier recovery point.
2) Restore the entire VM to an alternate location and copy the desired file(s) out of that restored VM.
3) Create a new AD-HOC recovery point manually. That will enable ILR to work for all recovery points including the latest one you made manually.UPDATE: This issue is now fixed in DPM 2012 R2 UR13. However, you need to consider the following for the fix to work.
1. For Protection Groups created after applying UR13, Item Level Recovery will work fine.
2. For older Protection Groups, you need to modify the PGs by adding/removing a small VM or making some configuration changes to the PG for the fix to work. - Link
-
Link
SC DPM 2016 brings in features that delivers improvements in multiple key areas of backup efficiency, performance, flexibility and security. With Modern Backup Storage (MBS), DPM 2016 is turning around the way backups are stored by leveraging modern technologies like ReFS Block Cloning, ReFS Allocate On Write and VHDX. This to 3X faster backups and 50% reduction in storage consumption thus reducing overall backup TCO.
https://blogs.technet.microsoft.com/dpm/2016/10/19/announcing-sc-dpm-2016-with-modern-backup-storage/ -
Link
DPM 2016 comes with Modern Backup Storage which delivers 50% storage savings, 3x faster backups, and more efficient storage with Workload-Aware storage. Modern Backup Storage can be used by setting up DPM 2016 on Windows Server 2016.
Add Storage to DPM 2016 -
Link
Very happy to report the release SC DPM 2012 R2 UR11 that adds protection for VMware host level backup of VMs. Here are major features of VMware VM Backup. Here are the few highlights of VMware VM backup.
- Agentless backup
- Backup at vCenter or ESXi level
- Backup VMware VM folders
- Ability to do ILR
- Protect large data center using DPM Scale Out
- Backup to Disk/Cloud
Here are the blogs and articles that are published.
BLOG - Implementing Microsoft DPM host level protection of VMware VMs
BLOG-Announcing VMware VM Backup using System Center Data Protection Manager
-
Link
We have made a bunch of improvements in this update, related to reboot requirements, Enterprise file server backup, online backups and more.
-
Link
After you install DPM 2012 R2 UR7 update (KB3065246), DPM cannot delete expired recovery points. This will lead to extra recovery point volume usage.
Please download and install the fix for this issue.
Fix for a known issue with Update Rollup 7 for System Center 2012 R2 Data Protection Manager
MORE INFORMATION
==============DPM 2012 R2 UR7 introduced a new switch to the powershell cmdlet Get-RecoveryPoint, it now requires a -Datasource switch as seen below:
Get-RecoveryPoint -Datasource $ds
If you have any custom powershell scripts that use that cmdlet, please modify them to use the new -Datasource switch.
-
Link
Below are some handy links for the DPM Console troubleshooter which can help you resolve console crashes and three DPMDB maintenance blogs.
Guided Walkthrough for Troubleshooting Data Protection Manager console issues
DPMDB Maintenance Part 1: Database consistency check and your DPMDB
DPMDB Maintenance Part 2: Identifying and dealing with fragmentation in your DPMDB
DPMDB Maintenance Part 3: Dealing with a large DPMDB -
Link
Hello - Please find the following BLOG on the new streamlined method of registering a DPM 2012 SP1 or later DPM server to backup to Azure. There is no longer a requirement to upload a certificate when registering a DPM server for online backups.
Below are two Blogs on the subject.
How to register a DPM 2012 SP1 or later server to backup to Microsoft Azure
Simplified workflow to configure backup to Azure from DPM -
Link
Please find the following whitepaper detailing how to dedup DPM storage pool to help reduce overall disk storage requirements for disk based backups. This solution requires Windows 2012 R2 and System Center Data Protection Manager 2012 R2
-
Link
For DPM Console crash issues, please review this troubleshooter before posting a new question.
Guided Walkthrough for Troubleshooting Data Protection Manager console issues
-
Link
updated 6-5-13
Hello Everyone,
System Center DPM 2012 is not supported running on Windows Server 2012. We have had reports of DPM 2012 console crashes when navigating to the recovery tab if DPM 2012 is installed on Windows server 2012. This issue is fixed in System Center 2012 SP1 which supports being installed on Windows 2012 operating system.
If you are already in this condition, you have two options:
A) You can upgrade to System Center 2012 Sp1.
B) You can backup the DPMDB, rebuild the DPM server using Windows 2008 R2 install DPM 2012, then restore your current database. The process is explained in the below technet article.
Recovering DPM servers
http://technet.microsoft.com/en-us/library/jj244606.aspxWe also do not support protecting Windows Server 2012 workloads using SCS DPM 2012, you must upgrade to DPM 2012 Sp1. Please do not expect any support for DPM 2012 running on Windows Server 2012, or issues while protecting data sources in Windows Server 2012.
