locked
Problem with DPM 2012 R2 console after install RU5 RRS feed

  • Question

  • Hello,

    I have DPM 2012 R2 server on production enviroment (OS Windows 2012 R2 Server). After install all updates from Windows Update (MS patch tuesday 2/10/2015) I have proble with DPM console. Console not started - only show window with product name and text Now loading .....

    Backup jobs working OK:

    Backup job for datasource: SQLC\PSESCoreTab on production server: SQL Server (MSSQLSERVER).FAFC.faf.cuni.cz completed succcessfully.

    Recovery point created at: 11. 2. 2015 12:00:11.

    Backup Type: disk backup.

    How I resolve this problém ? Uninstall RU5 ?

    On same server installed Operating manager agent RU5.

    Thanks,

    Snake AG

    Wednesday, February 11, 2015 5:30 PM

Answers

  • Hi Guys

    the GUI crashes issue after R5 in the following cases

    Case1:

    1. You have never protected the DPM server’s System State, with the DPM server itself
    2. You have protected some System State from other Production Servers
    3. You upgrade to UR5
    4. You trigger an inquiry on the DPM Server itself

    Case2:

    1. You have a primary DPM server protecting the System State of a Production Server
    2. You have attached the primary DPM server to a secondary DPM server for Disaster Recovery
    3. You upgrade the secondary DPM server to UR5
    4. You trigger an inquiry on the primary DPM server in the secondary DPM server UI

    Then:

    1. The UI crashes, and gets hung if you start it again
    2. Cmdlets are also unresponsive

    Workaround:

    1. Stop all DPM services (MSDPM, Access Manager, DPM Writer, DPMRA, VssNullProvider)
    2. Backup your DPM DB
    3. Open the SQL Server Management Studio (Either from Start Screen, or Run: ssms.exe)
    4. Connect to the SQL Instance holding the DPM DB
    5. Right click the DPM DB and click on “New Query”
    6. Paste the following SQL script:

    DELETE FROM dbo.tbl_IM_ProtectedObject

    WHERE ProtectedObjectId IN

    (

           SELECT PO.ProtectedObjectId

           FROM dbo.tbl_IM_ProtectedObject AS PO

           JOIN dbo.tbl_AM_Server AS SRVR

           ON PO.ServerId = SRVR.ServerId

           WHERE (CONVERT(VARCHAR(4096),PO.ComponentName) = 'System State' OR CONVERT(VARCHAR(4096),PO.ComponentName) = 'Bare Metal Recovery')

           AND PO.ProtectedInPlan = 0

           AND PO.ProtectedObjectId NOT IN (SELECT ProtectedObjectId FROM dbo.tbl_IM_ProtectedObjectAlerts)

           AND LOWER(CONVERT(XML,PO.LogicalPath).value(

                  '(/child::ArrayOfInquiryPathEntryType/child::InquiryPathEntryType[attribute::Type="Server"]/attribute::Value)[1]',

                  'VARCHAR(4096)'

           )) != LOWER(SRVR.ServerName)

    )

    1. Press F5 to execute
    2. Start all DPM services
    3. The UI and cmdlets should work normally as before

    Thanks Islam

     

    Islam Gomaa Microsoft MVP | System Center Data Protection Manager email: Islam @ Islam Gomaa.com Blog: http://owsug.ca/blogs/IslamGomaa Website: www.islamGomaa.com

    Friday, February 13, 2015 3:05 PM
  • Heuréka - problem solved :-)

    I must run script from Islam Gomaa AND remove backup DPM DB from protection group on the afected DPM server.


    • Marked as answer by SnakeAG Saturday, February 21, 2015 7:51 AM
    • Edited by SnakeAG Saturday, February 21, 2015 7:52 AM
    Saturday, February 21, 2015 7:51 AM
  • We have released a hotfix with a permanent fix for the DPM 2012 R2 UR5 Console Hang post release issue.

    The details are available at the below link:

    Console crashes when you run an inquiry on a DPM server after you install Update Rollup 5 for System Center 2012 R2 Data Protection Manager


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Friday, March 6, 2015 12:48 AM
    Moderator

All replies

  • Add on)

    DPM PowerShell not working:

    PS C:\Program Files\Microsoft System Center 2012 R2\DPM\DPM\bin> get-protectiongroup
    WARNING: Connecting to DPM server: name_deleted

    ... and hangup

    Thursday, February 12, 2015 6:11 AM
  • I have the same issue on one of the two DPM servers I have upgraded. DPM is doing backups just fine but the console is not working at all. It is stuck at "please wait while alerts are being retrieved". I cannot switch to any other part in the console at all. I have tried deleting the V2settings.xml file to no avail.
    Thursday, February 12, 2015 9:27 AM
  • Count another failed one here.  Console just hangs on launch.

    Glad this was just a test box and didn't apply this to my production systems.   

    When I try to close the hung console it says "You must close all dialog box before you can close the System Center 2012 R2 DPM Administrator Console".  Not sure if that is just the product window or similar to the Visual Studio patch issues as well.


    Thursday, February 12, 2015 2:22 PM
  • We are investigating the issue, could you please let us know which language the OS and DPM are?

    Moreover are you protecting BMR?

    Thursday, February 12, 2015 2:42 PM
    Moderator
  • OS a DPM language is English, Locale+keyboard is Czech

    (I have  DPM RU5 setup log)

    I not fully understand your second question ... I dont have BMR backup DPM server.

    Thursday, February 12, 2015 3:07 PM
  • OS and DPM are both US English. And I think the affected DPM server is not protecting BMR.
    Thursday, February 12, 2015 3:12 PM
  • Same problem here.

    Console hanging. DPM running in a VM

    Thursday, February 12, 2015 3:14 PM
  • Mine is Windows 2012 R2 Datacenter with DPM 2012 R2 RU5 running as a VM on Hyper-V 2008.

    English language... this one was only protecting a couple SQL workloads.

    I don't believe I had a BMR setup on there.  If there was it was an old inactive backup.  It was also backing up to Azure storage vault.

    Thursday, February 12, 2015 3:15 PM
  • Hello everyone,

    We are having a hard time reproducing this issue in-house, can you share the Operating System and DPM language you are running on the effected DPM Servers ?


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, February 12, 2015 3:20 PM
    Moderator
  • Something get hosed in a SQL table during update?

    Only error entries I see in install log are as follows:

    MSI (s) (84:80) [17:16:21:846]: Note: 1: 2262 2: Error 3: -2147287038 

    MSI (s) (84:80) [17:16:21:861]: Transforming table CustomAction.

    MSI (s) (84:80) [17:16:21:861]: Note: 1: 2262 2: CustomAction 3: -2147287038 
    MSI (s) (84:80) [17:16:21:861]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:861]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:861]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:21:861]: Transforming table Feature.
    MSI (s) (84:80) [17:16:21:908]: Transforming table CustomAction.

    MSI (s) (84:80) [17:16:21:908]: Note: 1: 2262 2: CustomAction 3: -2147287038 
    MSI (s) (84:80) [17:16:21:924]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:924]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:924]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:21:924]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:924]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:924]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:21:924]: Transforming table Feature.

    SI (s) (84:80) [17:16:21:986]: PROPERTY CHANGE: Adding ProductState property. Its value is '5'.
    MSI (s) (84:80) [17:16:21:986]: PROPERTY CHANGE: Adding ProductToBeRegistered property. Its value is '1'.
    MSI (s) (84:80) [17:16:21:986]: Package name retrieved from configuration data: 'msdpm.msi'
    MSI (s) (84:80) [17:16:21:986]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:986]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:986]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:21:986]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:986]: Transforming table Error.

    MSI (s) (84:80) [17:16:21:986]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:21:986]: Transforming table Error.

    MSI (s) (84:80) [17:16:22:002]: Transforming table Error.

    MSI (s) (84:80) [17:16:22:002]: Note: 1: 2262 2: Error 3: -2147287038 
    MSI (s) (84:80) [17:16:22:002]: Note: 1: 2262 2: AdminProperties 3: -2147287038 

    Thursday, February 12, 2015 3:29 PM
  • Add on)

    DPM PowerShell not working:

    PS C:\Program Files\Microsoft System Center 2012 R2\DPM\DPM\bin> get-protectiongroup
    WARNING: Connecting to DPM server: name_deleted

    ... and hangup

    I get the same result with Powershell as well.

    All services are running too.

    Thursday, February 12, 2015 3:35 PM
  • Initially I was able to open up to an empty console but that seems to no longer be possible after a couple reboots.
    Thursday, February 12, 2015 3:36 PM
  • Property(S): SystemLanguageID = 1029
    Property(S): ComputerName = BARNEY
    Property(S): UserLanguageID = 1029
    Property(S): UserSID = S-1-5-18
    Property(S): LogonUser = SYSTEM
    Property(S): MsiTrueAdminUser = 1
    Property(S): AdminUser = 1
    Property(S): VirtualMemory = 15732
    Property(S): PhysicalMemory = 16339
    Property(S): Intel = 6
    Property(S): Msix64 = 6
    Property(S): MsiAMD64 = 6
    Property(S): ShellAdvtSupport = 1
    Property(S): OLEAdvtSupport = 1
    Property(S): GPTSupport = 1
    Property(S): FontsFolder = C:\Windows\Fonts\
    Property(S): DesktopFolder = C:\Users\Public\Desktop\
    Property(S): StartMenuFolder = C:\ProgramData\Microsoft\Windows\Start Menu\
    Property(S): ProgramMenuFolder = C:\ProgramData\Microsoft\Windows\Start Menu\Programs\
    Property(S): StartupFolder = C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\
    Property(S): AdminToolsFolder = C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\
    Property(S): MyPicturesFolder = C:\Windows\system32\config\systemprofile\Pictures\
    Property(S): LocalAppDataFolder = C:\Windows\system32\config\systemprofile\AppData\Local\
    Property(S): CommonAppDataFolder = C:\ProgramData\
    Property(S): TemplateFolder = C:\ProgramData\Microsoft\Windows\Templates\
    Property(S): SendToFolder = C:\Windows\system32\config\systemprofile\AppData\Roaming\Microsoft\Windows\SendTo\
    Property(S): RecentFolder = C:\Windows\system32\config\systemprofile\AppData\Roaming\Microsoft\Windows\Recent\
    Property(S): PrintHoodFolder = C:\Windows\system32\config\systemprofile\AppData\Roaming\Microsoft\Windows\Printer Shortcuts\
    Property(S): PersonalFolder = C:\Windows\system32\config\systemprofile\Documents\
    Property(S): NetHoodFolder = C:\Windows\system32\config\systemprofile\AppData\Roaming\Microsoft\Windows\Network Shortcuts\
    Property(S): FavoritesFolder = C:\Windows\system32\config\systemprofile\Favorites\
    Property(S): AppDataFolder = C:\Windows\system32\config\systemprofile\AppData\Roaming\
    Property(S): CommonFiles64Folder = C:\Program Files\Common Files\
    Property(S): CommonFilesFolder = C:\Program Files (x86)\Common Files\
    Property(S): ProgramFilesFolder = C:\Program Files (x86)\
    Property(S): TempFolder = C:\Windows\TEMP\
    Property(S): RemoteAdminTS = 1
    Property(S): SystemFolder = c:\Windows\SysWOW64\
    Property(S): System64Folder = C:\Windows\system32\
    Property(S): WindowsVolume = c:\
    Property(S): WindowsFolder = c:\Windows\
    Property(S): MsiNTProductType = 3
    Property(S): ServicePackLevelMinor = 0
    Property(S): ServicePackLevel = 0
    Property(S): WindowsBuild = 9600
    Property(S): VersionNT64 = 603
    Property(S): ACTION = INSTALL
    Property(S): VersionNT = 603
    Property(S): VersionMsi = 5.00
    Property(S): VersionDatabase = 200
    Property(S): PRODUCTLANGUAGE = 1033
    Property(S): CLIENTPROCESSID = 1524
    MSI (s) (74:24) [21:07:50:606]: Product: Microsoft System Center 2012 R2 Data Protection Manager - Update 'Microsoft System Center 2012 R2 Data Protection Manager Update - KB3021791' installed successfully.

    MSI (s) (74:24) [21:07:50:778]: Windows Installer installed an update. Product Name: Microsoft System Center 2012 R2 Data Protection Manager. Product Version: 4.2.1205.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Update Name: Microsoft System Center 2012 R2 Data Protection Manager Update - KB3021791. Installation success or error status: 0.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 1728
    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:778]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Transforming table Error.

    MSI (s) (74:24) [21:07:50:793]: Note: 1: 2262 2: Error 3: -2147287038
    MSI (s) (74:24) [21:07:50:793]: Product: Microsoft System Center 2012 R2 Data Protection Manager -- Configuration completed successfully.

    MSI (s) (74:24) [21:07:50:793]: Windows Installer reconfigured the product. Product Name: Microsoft System Center 2012 R2 Data Protection Manager. Product Version: 4.2.1205.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Reconfiguration success or error status: 0.

    MSI (s) (74:24) [21:07:50:793]: Value of RebootAction property is 1
    MSI (s) (74:24) [21:07:50:793]: Windows Installer requires a system restart. Product Name: Microsoft System Center 2012 R2 Data Protection Manager. Product Version: 4.2.1205.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Type of System Restart: 2. Reason for Restart: 2.

    MSI (s) (74:24) [21:07:50:793]: Product: Microsoft System Center 2012 R2 Data Protection Manager. Restart required. The installation or update for the product required a restart for all changes to take effect.  The restart was deferred to a later time.

    MSI (s) (74:24) [21:07:50:793]: Attempting to delete file c:\Windows\Installer\730c2.msp
    MSI (s) (74:24) [21:07:50:871]: Deferring clean up of packages/files, if any exist
    MSI (s) (74:24) [21:07:50:871]: Attempting to delete file c:\Windows\Installer\730c2.msp
    MSI (s) (74:24) [21:07:50:871]: Unable to delete the file outside of the engine. LastError = 2
    MSI (s) (74:24) [21:07:50:871]: MainEngineThread is returning 3010
    MSI (s) (74:6C) [21:07:56:558]: RESTART MANAGER: Previously shut down applications have been restarted.
    MSI (s) (74:6C) [21:07:56:558]: RESTART MANAGER: Session closed.
    MSI (s) (74:6C) [21:07:56:558]: No System Restore sequence number for this installation.
    === Logging stopped: 10. 2. 2015  21:07:50 ===
    MSI (s) (74:6C) [21:07:56:558]: User policy value 'DisableRollback' is 0
    MSI (s) (74:6C) [21:07:56:558]: Machine policy value 'DisableRollback' is 0
    MSI (s) (74:6C) [21:07:56:558]: Incrementing counter to disable shutdown. Counter after increment: 0
    MSI (s) (74:6C) [21:07:56:558]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2
    MSI (s) (74:6C) [21:07:56:558]: Note: 1: 2265 2:  3: -2147287035
    MSI (s) (74:6C) [21:07:56:558]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2
    MSI (s) (74:6C) [21:07:56:558]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
    MSI (s) (74:6C) [21:07:56:558]: Post-install cleanup: removing installer file 'c:\Windows\Installer\730c2.msp'
    MSI (s) (74:6C) [21:07:56:589]: Destroying RemoteAPI object.
    MSI (s) (74:7C) [21:07:56:589]: Custom Action Manager thread ending.
    MSI (c) (F4:8C) [21:07:56:589]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
    MSI (c) (F4:8C) [21:07:56:589]: MainEngineThread is returning 3010
    MSI (c) (F4:A8) [21:07:56:604]: RESTART MANAGER: Previously shut down applications have been restarted.
    MSI (c) (F4:A8) [21:07:56:604]: RESTART MANAGER: Session closed.
    === Verbose logging stopped: 10. 2. 2015  21:07:56 ===

    Thursday, February 12, 2015 3:40 PM
  • Hello everyone,

    We are having a hard time reproducing this issue in-house, can you share the Operating System and DPM language you are running on the effected DPM Servers ?


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Are going to need some DPM Logs?
    Thursday, February 12, 2015 4:17 PM
  • I haven't had a chance to look through my DPM logs yet

    \Program Files\Microsoft System Center 2012 R2\DPM\DPM\Temp

    Thursday, February 12, 2015 4:25 PM
  • I'll check back in a few days.  I may just need to restore my last backup and wait this one out.  I downloaded and re-applied the RU but of course that completely successfully and just left me with same issue.  So happy this was only a test server.  I feel for you guys dealing with this on a production box.
    • Edited by Muzzly1298 Thursday, February 12, 2015 5:00 PM
    Thursday, February 12, 2015 4:59 PM
  • We are running DPM 2012R2 English on Server 2012R2.  Same problem after letting Windows Update apply RU5.  I also manually ran the update after and that didn't fix it.  Initially, the Console would open and ask me to wait while it retrieved data.  After a server reboot it just sits at the DPM splash screen saying "Now loading..."  I have to kill the task.

    Watching processes, TCP view, and SQL Server logs, it looks like the syncs and restore points are working.

    No errors are being logged in the SQL logs or Windows Event viewer other than the one when I kill the task.

    TCP View shows 100 connections established between mmc.exe and sqlservr.exe using different ports for mmc.exe but the same port on the sqlservr.exe side.  I'm not sure, but I don't think that is normal.

    Thursday, February 12, 2015 8:36 PM
  • I have two DPM servers that I updated yesterday morning. I always do one first, make sure the console works before I update the other because I have bitten before with this same problem.

    Used both of them throughout the day yesterday, and updated the protection agents last night since they required a reboot. No problems. Now I need to remove an agent, and both of them I cannot open the console.

    Both OS and DPM are US English.

    Thursday, February 12, 2015 9:15 PM
  • @pleiadesalcyone Question-did you install RU5 yesterday and then install other Feb 10 updates later? Might it be another new update interfering?
    Thursday, February 12, 2015 11:03 PM
  • Hello,

    I have the same issue.

    OS English US.

    It is not possible to open the snap-in. The MMC starts. I can see some menu and help.

    It is not possible to close the console. I have to use the task manager. It ask to close the window first (which I can't see).

    It is also not possible to open the console from the workstation (remote) which is not updated yet.

    The errors on the workstation are the same.

    Friday, February 13, 2015 7:22 AM
  • same her , Windows Server 2008 R2 SQL 2008 R2 
    Friday, February 13, 2015 8:27 AM
  • Same here DPM 2012R2 installed on Windows 2012R2 everything is installed as "en-US". The console cannot start while the back up processes are still being executed.
    Friday, February 13, 2015 9:52 AM
  • @pleiadesalcyone Question-did you install RU5 yesterday and then install other Feb 10 updates later? Might it be another new update interfering?

    No, all the updates at the same time. I updated one server before the other one. I think it was DPM 2010 that I ran into the same problem of an update causing the console to not open. Now I update one and make sure everything works, then update the other when I see a update rollup.

    That is what makes this one strange, I updated the protection agent on the servers I could restart during the day. Then that night, I updated the protection agent on the servers that I could only restart after hours. I think the console was useable for at least 12 hours before it broke. I updated the first server at 8AM, finished updating the protection agents around 8-9PM, then the next day around 2-3PM I couldn't get into the console.

    Friday, February 13, 2015 1:26 PM
  • Hi Guys

    the GUI crashes issue after R5 in the following cases

    Case1:

    1. You have never protected the DPM server’s System State, with the DPM server itself
    2. You have protected some System State from other Production Servers
    3. You upgrade to UR5
    4. You trigger an inquiry on the DPM Server itself

    Case2:

    1. You have a primary DPM server protecting the System State of a Production Server
    2. You have attached the primary DPM server to a secondary DPM server for Disaster Recovery
    3. You upgrade the secondary DPM server to UR5
    4. You trigger an inquiry on the primary DPM server in the secondary DPM server UI

    Then:

    1. The UI crashes, and gets hung if you start it again
    2. Cmdlets are also unresponsive

    Workaround:

    1. Stop all DPM services (MSDPM, Access Manager, DPM Writer, DPMRA, VssNullProvider)
    2. Backup your DPM DB
    3. Open the SQL Server Management Studio (Either from Start Screen, or Run: ssms.exe)
    4. Connect to the SQL Instance holding the DPM DB
    5. Right click the DPM DB and click on “New Query”
    6. Paste the following SQL script:

    DELETE FROM dbo.tbl_IM_ProtectedObject

    WHERE ProtectedObjectId IN

    (

           SELECT PO.ProtectedObjectId

           FROM dbo.tbl_IM_ProtectedObject AS PO

           JOIN dbo.tbl_AM_Server AS SRVR

           ON PO.ServerId = SRVR.ServerId

           WHERE (CONVERT(VARCHAR(4096),PO.ComponentName) = 'System State' OR CONVERT(VARCHAR(4096),PO.ComponentName) = 'Bare Metal Recovery')

           AND PO.ProtectedInPlan = 0

           AND PO.ProtectedObjectId NOT IN (SELECT ProtectedObjectId FROM dbo.tbl_IM_ProtectedObjectAlerts)

           AND LOWER(CONVERT(XML,PO.LogicalPath).value(

                  '(/child::ArrayOfInquiryPathEntryType/child::InquiryPathEntryType[attribute::Type="Server"]/attribute::Value)[1]',

                  'VARCHAR(4096)'

           )) != LOWER(SRVR.ServerName)

    )

    1. Press F5 to execute
    2. Start all DPM services
    3. The UI and cmdlets should work normally as before

    Thanks Islam

     

    Islam Gomaa Microsoft MVP | System Center Data Protection Manager email: Islam @ Islam Gomaa.com Blog: http://owsug.ca/blogs/IslamGomaa Website: www.islamGomaa.com

    Friday, February 13, 2015 3:05 PM
  • Hi Guys

    the GUI crashes issue after R5 in the following cases

    Case1:

    1. You have never protected the DPM server’s System State, with the DPM server itself
    2. You have protected some System State from other Production Servers
    3. You upgrade to UR5
    4. You trigger an inquiry on the DPM Server itself

    Case2:

    1. You have a primary DPM server protecting the System State of a Production Server
    2. You have attached the primary DPM server to a secondary DPM server for Disaster Recovery
    3. You upgrade the secondary DPM server to UR5
    4. You trigger an inquiry on the primary DPM server in the secondary DPM server UI

    Then:

    1. The UI crashes, and gets hung if you start it again
    2. Cmdlets are also unresponsive

    Workaround:

    1. Stop all DPM services (MSDPM, Access Manager, DPM Writer, DPMRA, VssNullProvider)
    2. Backup your DPM DB
    3. Open the SQL Server Management Studio (Either from Start Screen, or Run: ssms.exe)
    4. Connect to the SQL Instance holding the DPM DB
    5. Right click the DPM DB and click on “New Query”
    6. Paste the following SQL script:

    DELETE FROM dbo.tbl_IM_ProtectedObject

    WHERE ProtectedObjectId IN

    (

           SELECT PO.ProtectedObjectId

           FROM dbo.tbl_IM_ProtectedObject AS PO

           JOIN dbo.tbl_AM_Server AS SRVR

           ON PO.ServerId = SRVR.ServerId

           WHERE (CONVERT(VARCHAR(4096),PO.ComponentName) = 'System State' OR CONVERT(VARCHAR(4096),PO.ComponentName) = 'Bare Metal Recovery')

           AND PO.ProtectedInPlan = 0

           AND PO.ProtectedObjectId NOT IN (SELECT ProtectedObjectId FROM dbo.tbl_IM_ProtectedObjectAlerts)

           AND LOWER(CONVERT(XML,PO.LogicalPath).value(

                  '(/child::ArrayOfInquiryPathEntryType/child::InquiryPathEntryType[attribute::Type="Server"]/attribute::Value)[1]',

                  'VARCHAR(4096)'

           )) != LOWER(SRVR.ServerName)

    )

    1. Press F5 to execute
    2. Start all DPM services
    3. The UI and cmdlets should work normally as before

    Thanks Islam

     

    Islam Gomaa Microsoft MVP | System Center Data Protection Manager email: Islam @ Islam Gomaa.com Blog: http://owsug.ca/blogs/IslamGomaa Website: www.islamGomaa.com


    This looks like it is deleting a record that points to what should be a system state protection for the DPM server but must not exist.  Is that true?
    • Proposed as answer by pywong Monday, February 16, 2015 1:08 AM
    • Unproposed as answer by pywong Monday, February 16, 2015 1:08 AM
    Friday, February 13, 2015 8:57 PM
  • That does delete the duplicate entry


    Islam Gomaa Microsoft MVP | System Center Data Protection Manager email: Islam @ Islam Gomaa.com Blog: http://blog.islamGomaa.com

    Saturday, February 14, 2015 3:11 PM
  • Thanks for workaround !

    Snake AG

    Saturday, February 14, 2015 6:08 PM
  • The SQL Script did it! Console and PS responsive again.

    Oh my god ...


    Marc

    Sunday, February 15, 2015 9:56 AM
  • The SQL script did the tricks for my case. Thanks!
    Monday, February 16, 2015 1:10 AM
  • Big thnx!!!
    Monday, February 16, 2015 1:37 AM
  • Whilst there is a workaround, this is not acceptable, customers are becoming very cautious with all Microsoft updates of late and rightly so.  As an organisation Microsoft need to significantly raise their game and address the serious issue of why the quality of updates and general patches are being released with schoolboy errors that should have been detected.

    Carl Smith MCITP-EA

    Monday, February 16, 2015 9:08 AM
  • I run SQL script on Sunday and console worked OK.

    Unfortunately, I tried to start console today and again the same problem.

    Snake AG

    Monday, February 16, 2015 12:41 PM
  • I ran the script and all seems well.  Thanks for the workaround!!

    I did have something else peculiar that may or may not be related.  This is a new setup for us, so DPM is allowed to automatically grow the volumes.  At the same time that I could not open the GUI or PS, my DPM server stopped automatically growing volumes for the recovery points of my protected clients.  I started getting space warnings and failed recovery points.  After I ran the script and was able to open the GUI again, the volumes automatically grew on the recovery point creation.  Maybe it uses PS to grow the volumes and that was failing?

    Monday, February 16, 2015 4:44 PM
  • The same problem here... :-(

    >I run SQL script on Sunday and console worked OK.

    >Unfortunately, I tried to start console today and again the same problem.

    Tuesday, February 17, 2015 6:10 AM
  • Hi,

    the script has help first, but after one day I see the same problem again

    First I could see the MMC without the date. After reboot the MMC stays by loading the data again.

    The Problem is not solved with the script permanentlly.

    Tuesday, February 17, 2015 6:54 AM
  • I guess I won't even bother trying this specific workaround it if only lasts for a day.


    Tuesday, February 17, 2015 1:18 PM
  • Do not install RU5 is the recommendation!

    Wait for a hotfix from MS.

    Tuesday, February 17, 2015 2:19 PM
  • As expected from what others saw I am able to reopen my console.  I guess I will see if problem comes back in 24 hours.  
    Tuesday, February 17, 2015 2:34 PM
  • Do not install RU5 is the recommendation!

    Wait for a hotfix from MS.

    Believe me... RU5 won't be going onto any of my production DPM servers until there is a hotfix or re-release of patched RU5 update.  I was hoping to take advantage of the additional Azure supported workloads too.  
    Tuesday, February 17, 2015 3:00 PM
  • Comments from MS?

    Is it possible to uninstall RU5?

    Wednesday, February 18, 2015 6:01 AM
  • Dear Anurag and Mike (MSFT),

    do you have any information how to resolve this problem ?

    I am ready to allow remote access to the DPM server affected by this issue (via support.microsoft.com/help)

    Thanks,

    Snake AG

    Wednesday, February 18, 2015 6:12 AM
  • Hi Guys

    I ran the script on my servers and I can now proceed to access the mmc console for DPM and view all of my protection groups - but I have noticed another issue. When trying to expand my primary DPM server from the secondary DPM it cannot enumerate the primary servers protected servers list.

    I can see in event viewer that I get error 945 on MSDPM as below: 



    The description for Event ID 945 from source MSDPM cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

    If the event originated on another computer, the display information had to be saved with the event.

    The following information was included with the event: 

    Unable to connect to the DPM database because of a general database failure.  Make sure that SQL Server is running and that it is configured correctly.

    Problem Details:
    <FatalServiceError><__System><ID>19</ID><Seq>30</Seq><TimeCreated>17/02/2015 22:22:59</TimeCreated><Source>DpmThreadPool.cs</Source><Line>163</Line><HasError>True</HasError></__System><ExceptionType>SqlException</ExceptionType><ExceptionMessage>Invalid object name '##06239356_ac26_4c11_bc0e_abc717754efatmp_tbl_ProtectedPS'.</ExceptionMessage><ExceptionDetails>System.Data.SqlClient.SqlException (0x80131904): Invalid object name '##06239356_ac26_4c11_bc0e_abc717754efatmp_tbl_ProtectedPS'.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
       at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean&amp; dataReady)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task&amp; task, Boolean asyncWrite, SqlDataReader ds)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task&amp; task, Boolean asyncWrite)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Internal.EnterpriseStorage.Dls.DB.SqlRetryCommand.ExecuteNonQuery()
       at Microsoft.Internal.EnterpriseStorage.Dls.AgentManager.DB.AMServerInfo.GenerateProtectedPSServerIDs(SqlContext ctx)
       at Microsoft.Internal.EnterpriseStorage.Dls.AgentManager.DB.ProtectedPSs.GeneratePSInfoXml(Object obj)
    ClientConnectionId:b7a20208-7ed9-4ad5-8381-2f2a258f0dfa
    Error Number:208,State:0,Class:16</ExceptionDetails></FatalServiceError>


    the message resource is present but the message is not found in the string/message table
    So it cannot find this table perhaps...

    <GUID>tmp_tbl_ProtectedPS


    I have found a thread where someone else is experiencing a similar issue (However it is not related to RU5):


    Maybe I need to run a clean up script on the DPM DB as suggested in the previous thread??
    • Edited by NeilRudds Wednesday, February 18, 2015 8:13 AM spelling
    Wednesday, February 18, 2015 8:09 AM
  • My test server survived the 24 hour window and can also enumerate the servers being backed up.  I am only using it to backup some SQL though.

    Until I hear this issue is fully addressed I still plan to hold off on installing RU5 on production servers.  Hoping further info comes to light within the month or sooner.

    Wednesday, February 18, 2015 8:07 PM
  • the script is  a permanent fix,


    Islam Gomaa Microsoft MVP | System Center Cloud Datacenter Management email: Islam @ Islam Gomaa.com Blog: http://blog.islamGomaa.com

    Thursday, February 19, 2015 11:52 AM
  • No, the problem comes back, every day.:-(

    Thursday, February 19, 2015 5:04 PM
  • I was also unable to connect to my DPM server with the console after RU5 was installed.  I just ran Islam's SQL query and it immediately returned to what seems like normal.  Hopefully it stays that way.

    Thank you Islam Gomaa.

    Alex


    • Edited by alx9r Friday, February 20, 2015 1:10 AM
    Friday, February 20, 2015 1:10 AM
  • I must run this script every day.

    Why ?

    I backup DPM SQL DB on the same DPM server in one protection group - is this problem ?

    Snake AG

    Friday, February 20, 2015 6:00 AM
  • Same for me.

    I have two DPM-servers that take local backup of DPMDB every day, I remove that DB now from the protectiongroup, just a test.

    Friday, February 20, 2015 8:09 AM
  • I have a new installed DPM 2012 R2 server with RU5 running with no problem, no backup of the DPMDB.

    Now I tried to add the DPMDB to a protectiongroup, the MMC krasch and after that the Console don´t start.

    The fix works, but their is problem with local DBM/DB backup...

    Friday, February 20, 2015 8:24 AM
  • On my installation Islam Gomaa's delete query solves the console crash problem, but the solution is not permanent.  The issue returned after a day.  Running the delete query again solved the problem again.

    Alex

    Friday, February 20, 2015 10:11 PM
  • After running the script it solved the console problem. Then the next day the console issue came back. Need a permanent fix.


    Daniel Wingfield

    Saturday, February 21, 2015 7:42 AM
  • Heuréka - problem solved :-)

    I must run script from Islam Gomaa AND remove backup DPM DB from protection group on the afected DPM server.


    • Marked as answer by SnakeAG Saturday, February 21, 2015 7:51 AM
    • Edited by SnakeAG Saturday, February 21, 2015 7:52 AM
    Saturday, February 21, 2015 7:51 AM
  • OMG... CU2 all over again.... :(

    I'm also effected with the same issue here today... :(

    Hello Microsoft...?? Time to pull CU5 off until fixed?

    The DPM nightmare continues... :(

    Chris

    Sunday, February 22, 2015 7:27 PM
  • Hello folks -  Quick update on this thread. We are very thankful to Islam Gomaa for posting a script workaround which is working for some of you.  We are working on a releasing a product level fix and expect to get it out in the next 2 weeks. 

    Regards,

    Monday, February 23, 2015 12:17 PM
  • Thank You Islam Gomaa :)

    ---

    Some information about my setup:

    • Issue only on the secondary DPM (Server 2012 R2, DPM 2012 R2 ru5, language: German)
    • Three primary DPM (2x Server 2012 R2, 1x Server 2012, all DPM 2012 R2 ru 5, all language: German)

    The secondary DPM replicates VMs, Bare Metal, SQL, Exchange, File from the three primary DPM and it backs up the DPM Database from the primary DPMs.

    Secondary DPM got a Tape Library attached

    ---

    Something unusual (maybe related - maybe not):

    After I migrated my Hyper-V Clusters from Server 2012 to Server 2012 R2, I had 4 out of 7 old Hyper-V Hosts left on the secondary DPM. The Hosts were removed from primary DPM, but I am unable to remove the orphaned hosts from the secondary DPM.


    This posting is provided AS IS with no warranties.


    • Edited by Dark Grant Monday, February 23, 2015 1:04 PM
    Monday, February 23, 2015 1:01 PM
  • Hello folks -  Quick update on this thread. We are very thankful to Islam Gomaa for posting a script workaround which is working for some of you.  We are working on a releasing a product level fix and expect to get it out in the next 2 weeks. 

    Regards,

    First off, thank you Islam Gomaa for fixing this when not even Microsoft could.

    Secondly, maybe Microsoft shouldn't force the update on people as a required update when it clearly wasn't.  Also, don't you have Quality Assurance Testing done on your patches, or do you just create them and make them public?  It's ridiculous that this "patch" was even released - and not even as an optional update!  I am looking forward to the "fix" you release for this which will probably require a reboot of every server that has the DPM Agent installed AGAIN!  Maybe it's time to start looking for a different backup solution from a different company.

    Thursday, February 26, 2015 8:00 PM
  • I've got a single DPM server, bare metal setup for one production server, and this happened out of the blue here as well. The SQL script fixed things fine, and I'm waiting to see if things will remain operational (thanks to mr. Gomaa).

    I can say I'm, yet again, very disappointed in the way these failures get introduced, and I hope that the way this bug will be addressed won't be a repeat of the dreadful 'CSV I/O paused'-issue of 2012, that took Microsoft over 6 months to fix..

    Friday, February 27, 2015 10:10 AM
  • I've also fallen victim to this update...... not a good position to be in :(
    Tuesday, March 3, 2015 8:34 AM
  • Come on MS, it's been good few weeks now! I wouldn't like to use the above workarounds and have the issue fixed properly! Is the any any update on this?
    Tuesday, March 3, 2015 1:29 PM
  • Hi,

    Yes, we are in the process of validating a new fix to address this issue, as soon as we get it validated we will make it available as a permanent fix.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, March 3, 2015 8:26 PM
    Moderator
  • Hi,

    Yes, we are in the process of validating a new fix to address this issue, as soon as we get it validated we will make it available as a permanent fix.

    Hi Mike,

    That's great news. Problem is that this kind of software needs to be accessible at all times. Luckily I did not need to perform any restores since I lost access to console.

    By this Friday I need to regain access to console to make sure I have got enough tapes in my libraries.

    IS IT POSSIBLE TO UNINSTALL CU5 KB3021791?

    Thanks

    Wednesday, March 4, 2015 8:44 AM
  • Hi,

    The workaround SQL Script should get you back in the console - so there is no need to revert back to pre-UR5 state.  If you run into the issue again before the permanent fix is released, then just re-run the script.  I know this is an inconvenience, but the permanent fix should be available soon.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Wednesday, March 4, 2015 6:36 PM
    Moderator
  • there are other problems with ur5 as well, SharePoint item lvl recovery doesn't work either (Error 32017 0x80004003) MS should make a detailed how to for deinstall.
    Wednesday, March 4, 2015 10:39 PM
  • Hi,

    The workaround SQL Script should get you back in the console - so there is no need to revert back to pre-UR5 state.  If you run into the issue again before the permanent fix is released, then just re-run the script.  I know this is an inconvenience, but the permanent fix should be available soon.

    Hi Mike,

    The SQL Query has modified 22 rows in the DB but my console still does not open and gets stuck on "Now loading..."

    I really need to have this fixed. It's so enjoying that Microsoft allows for this sort of issues to go through quality control procedures in a first place and then ignore users for weeks after issue gets identified!!!

    Marian

    Thursday, March 5, 2015 9:48 AM
  • Thanks Islam - this worked perfectly. You saved my bacon (and my server)!
    Thursday, March 5, 2015 1:58 PM
  • We have released a hotfix with a permanent fix for the DPM 2012 R2 UR5 Console Hang post release issue.

    The details are available at the below link:

    Console crashes when you run an inquiry on a DPM server after you install Update Rollup 5 for System Center 2012 R2 Data Protection Manager


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Friday, March 6, 2015 12:48 AM
    Moderator