none
DPMRA Agent service fails to start

    Question

  • Hello.

    I have a problem with starting the DPMRA service after I've successfully installed the agent.

    The service fails to start with an error 1168. I've installed the agent successfully on numerous computers, only this one is giving me troubles.

    What could be the problem? I've tried rebooting the server, uninstalling and reinstalling the agent (including VC++ redist), but to no avail.

    Details: DPM 2010 RTM, protected machine is Windows 2008 R2 Std

    Here's the XML output of the eventlog entry:

     

     <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

    - <System>

      <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" /> 

      <EventID Qualifiers="49152">7024</EventID> 

      <Version>0</Version> 

      <Level>2</Level> 

      <Task>0</Task> 

      <Opcode>0</Opcode> 

      <Keywords>0x8080000000000000</Keywords> 

      <TimeCreated SystemTime="2010-08-30T14:26:18.679023300Z" /> 

      <EventRecordID>7075</EventRecordID> 

      <Correlation /> 

      <Execution ProcessID="492" ThreadID="1872" /> 

      <Channel>System</Channel> 

      <Computer>COMPUTER (original name removed)</Computer> 

      <Security /> 

      </System>

    - <EventData>

      <Data Name="param1">DPMRA</Data> 

      <Data Name="param2">%%1168</Data> 

      </EventData>

      </Event>

     

    Monday, August 30, 2010 2:36 PM

Answers

  • It sounds like the WMI or other expected exceptions are not in the firewall settings. This affects us even if you aren't using the firewall.

    Run the following commands from an elevated command prompt:

    • netsh advfirewall firewall set rule group=\"@FirewallAPI.dll,-29502\" new enable=yes
    • netsh advfirewall firewall set rule group=\"@FirewallAPI.dll,-34251\" new enable=yes
    • netsh advfirewall firewall add rule name=dpmra dir=in program=\"%PROGRAMFILES%\\Microsoft Data Protection Manager\\DPM\\bin\\DPMRA.exe\" profile=Any action=allow
    • netsh advfirewall firewall add rule name=DPMRA_DCOM_135 dir=in action=allow  protocol=TCP localport=135 profile=Any

    If either of the first two commands result in errors some default FW rules are missing and must be created.

    1. Copy the text below to a .reg file and merge the entries into the registry.
    2. Restart the Windows Firewall service.
    3. Run the above netsh again commands.
    4. If they succeed then run setdpmserver.exe.

    NOTE: The following registry entries may need to be v2.10 or v2.0. Check the existing entries in that registry location and you will be fine.

    Windows Registry Editor Version 5.00

    [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\FirewallRules]

    "RemoteSvcAdmin-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=RPC|App=%SystemRoot%\\system32\\services.exe|Name=@FirewallAPI.dll,-29503|Desc=@FirewallAPI.dll,-29506|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "RemoteSvcAdmin-NP-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=445|App=System|Name=@FirewallAPI.dll,-29507|Desc=@FirewallAPI.dll,-29510|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "RemoteSvcAdmin-RPCSS-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=RPC-EPMap|App=%SystemRoot%\\system32\\svchost.exe|Svc=RPCSS|Name=@FirewallAPI.dll,-29515|Desc=@FirewallAPI.dll,-29518|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "WMI-RPCSS-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=135|App=%SystemRoot%\\system32\\svchost.exe|Svc=rpcss|Name=@FirewallAPI.dll,-34252|Desc=@FirewallAPI.dll,-34253|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-WINMGMT-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|App=%SystemRoot%\\system32\\svchost.exe|Svc=winmgmt|Name=@FirewallAPI.dll,-34254|Desc=@FirewallAPI.dll,-34255|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-WINMGMT-Out-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=Out|Protocol=6|App=%SystemRoot%\\system32\\svchost.exe|Svc=winmgmt|Name=@FirewallAPI.dll,-34258|Desc=@FirewallAPI.dll,-34259|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-ASYNC-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|App=%systemroot%\\system32\\wbem\\unsecapp.exe|Name=@FirewallAPI.dll,-34256|Desc=@FirewallAPI.dll,-34257|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    /Steve


    Steve L [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, September 01, 2010 11:49 AM
    Moderator
  • Markos,

    By defualt the agent installation piece expects certain firewall exceptions to be in place regardless if the firewall service is running or not. Typically these exceptions revolve around the built-in WMI exceptions. If these exceptions are missing then we'll either have agent installation/configuration issues or communication problems. I am not on the team that supports the firewall but I do know that event when not in use there are some instances where it comes into play.

    In most cases with Windows Server 2008 and 2008 R2 we have the needed exceptions in place. However due to build process or role/feature additions or removals the needed exceptions are not in place. So part of troubleshooting the software firewall problems is we check for the default exceptions.

    If there are external firewalls, ISA/TMG and the like, and we have VPN tunnels between DPM and protected servers, there are other things that prevent traffic between the two. Even when allow all rules are in place some DPM RPC/DCOM traffic is dropped. So there are things we have to configure in that case.

    One more thing to take into account is that not all protected server interactios with the DPM server are going to be originated on the DPM side. We'll have DPMRA sessions originating from the protected server in response to DPM intitiated traffic. So we need to allow all in both directions.

    To your question about why, if all servers get the same firewall settings via GPOs, do some not work, my first step is the firewall exceptions. Then go from there.

    /Steve


    Steve L [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by MarkosP Thursday, September 09, 2010 6:09 AM
    Wednesday, September 08, 2010 9:55 AM
    Moderator

All replies

  • To add more details, when I try to manually attach the agent via setdpmserver.exe (setdpmserver -dpmservername <server>), it will return:

    "SetDpmServer failed with errorcode =0x80004005, error says: Unspecified error"

    Tuesday, August 31, 2010 6:57 AM
  • HI

    Verify you COM+ rights for the DPM agent. It could be that there is a missmatch in the rights.

    Click START / Administrative Tools / Component Services

    Expand Component Services / Computers / My Computer / DCOM Config

    Right click on the "DPM RA Services" and choose properties. Verify that under the Security tab / Launch and Activation Permissions you got Customize marked, click the Edit... button in the ACL you should see the computer account for your DPM server. If it's not present add it and mark all allow boxes and try to start the service again.

    BR

    Robert Hedblom


    Check out my DPM blog @ http://robertanddpm.blogspot.com
    Tuesday, August 31, 2010 8:38 AM
    Moderator
  • Thanks for the reply Robert, I've verified the DCOM permissions on DPM "RA Services" and they are correct, ie. Administrators, SYSTEM and the DPM server computer account all have full access.

    Any other ideas?

    Tuesday, August 31, 2010 8:57 AM
  • Verify that the communication ports for the DPM agent isn't used for some other application.

    Here's the port list DPM uses:

    http://technet.microsoft.com/en-us/library/ff399341.aspx

     


    Check out my DPM blog @ http://robertanddpm.blogspot.com
    Tuesday, August 31, 2010 9:45 AM
    Moderator
  • None app. that I know of. The only thing that is different from other servers is that this one has Firebird 2.1 installed. But that should not collide with DPM Agent or? I've tried stopping the Firebird server service before starting DPMRA service, but that didn't help either.

    All communication between the DPM server and the protected machine is allowed by a firewall rule (enforced in GPO, so it's the same for every server). I've also tried turning the firewall off.

    Otherwise there are no other apps listening on ports that DPM used - ofcourse except for those common ones like DNS, SMB etc.

    Tuesday, August 31, 2010 11:35 AM
  • I have a similar problem; but mine has knocked out our entire DPM backend... needless to say, our "powers that be" are not impressed; and I dont blame them. here is where I am currently at: ******************************************************************************* Last wednesday, we had something go horribly, horribly wrong with our DPM.. Now all of our backups are failing, none of the agents can be contacted, and furthermore, Restores are not able to be restored due to the agents not being able to be contacted. The first hint of a problem came after a reboot of the server, the console was denied access to the application itself. Even new installs of a new agent to a new computer fail; it is like somehow our MainDPM server has lost its security info. I dont even know where to start; it might be easier if somebody from MS asks me some specific questions. I have been done the road of making sure that Launch permissions and access permissions are on the COM object, so needless to say we are pretty frustrated. I have about a months worth of backups that I dont want to lose, however, I think we are looking at blowing away the entire server...but now my confidence is shot...How on earth can you depend on a technology that has a single point of failure (the agent) with something so critical as backups? I cant even roll over protection to the secondary DPM server because of comm issues with the primary, which I thought strange... regardless of the condition of the primary DPM, I should be able to do restores from the secondary, or at least start protecting the servers from the secondary after rolling over, but this is not the case...
    Tuesday, August 31, 2010 10:06 PM
  • It sounds like the WMI or other expected exceptions are not in the firewall settings. This affects us even if you aren't using the firewall.

    Run the following commands from an elevated command prompt:

    • netsh advfirewall firewall set rule group=\"@FirewallAPI.dll,-29502\" new enable=yes
    • netsh advfirewall firewall set rule group=\"@FirewallAPI.dll,-34251\" new enable=yes
    • netsh advfirewall firewall add rule name=dpmra dir=in program=\"%PROGRAMFILES%\\Microsoft Data Protection Manager\\DPM\\bin\\DPMRA.exe\" profile=Any action=allow
    • netsh advfirewall firewall add rule name=DPMRA_DCOM_135 dir=in action=allow  protocol=TCP localport=135 profile=Any

    If either of the first two commands result in errors some default FW rules are missing and must be created.

    1. Copy the text below to a .reg file and merge the entries into the registry.
    2. Restart the Windows Firewall service.
    3. Run the above netsh again commands.
    4. If they succeed then run setdpmserver.exe.

    NOTE: The following registry entries may need to be v2.10 or v2.0. Check the existing entries in that registry location and you will be fine.

    Windows Registry Editor Version 5.00

    [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\FirewallRules]

    "RemoteSvcAdmin-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=RPC|App=%SystemRoot%\\system32\\services.exe|Name=@FirewallAPI.dll,-29503|Desc=@FirewallAPI.dll,-29506|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "RemoteSvcAdmin-NP-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=445|App=System|Name=@FirewallAPI.dll,-29507|Desc=@FirewallAPI.dll,-29510|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "RemoteSvcAdmin-RPCSS-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=RPC-EPMap|App=%SystemRoot%\\system32\\svchost.exe|Svc=RPCSS|Name=@FirewallAPI.dll,-29515|Desc=@FirewallAPI.dll,-29518|EmbedCtxt=@FirewallAPI.dll,-29502|Edge=FALSE|"

    "WMI-RPCSS-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|LPort=135|App=%SystemRoot%\\system32\\svchost.exe|Svc=rpcss|Name=@FirewallAPI.dll,-34252|Desc=@FirewallAPI.dll,-34253|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-WINMGMT-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|App=%SystemRoot%\\system32\\svchost.exe|Svc=winmgmt|Name=@FirewallAPI.dll,-34254|Desc=@FirewallAPI.dll,-34255|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-WINMGMT-Out-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=Out|Protocol=6|App=%SystemRoot%\\system32\\svchost.exe|Svc=winmgmt|Name=@FirewallAPI.dll,-34258|Desc=@FirewallAPI.dll,-34259|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    "WMI-ASYNC-In-TCP"="v2.0|Action=Allow|Active=TRUE|Dir=In|Protocol=6|App=%systemroot%\\system32\\wbem\\unsecapp.exe|Name=@FirewallAPI.dll,-34256|Desc=@FirewallAPI.dll,-34257|EmbedCtxt=@FirewallAPI.dll,-34251|Edge=FALSE|"

    /Steve


    Steve L [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, September 01, 2010 11:49 AM
    Moderator
  • Well, thanks for the reply, but we have moved on past this.  We have the Primary DPM server being backed up in turn by a secondary DPM server.  When the primary is up, all comms to the Primary DPM server AND ANY server that had the Primary DPM server set as their primary DPM server, were not able to be communicated with.

    The mjnute we bring down the Pri DPM server, we are able to communicate with the agents on the boxes that were associated with the primary DPM server, thus now allowing us to "switch protection" over to the secondary. This is resulting in a huge data xfer and sync, but once we get that all caught up, we are going to blow away the primary DPM and rebuild from scratch....

    One thing to note, is that we started the process of DPM with the 2007 eval, then the 2007 RCeval, then onto the RTM.  After this crash, my first indication of a problem was after a restart of the Pri DPM, when firing up the console, I got "ACCESS DENIED"... I tried starting and stopping the DPM services, and got access denied.  Then i noticed that I had 3 sets of SQL database services, but none loading.  They were not running, and when i tried to manually start them using the default MICROSOFT$DPM$ACCT, I got ACCESS DENIED.  Very strange. 

    When I changed those over to Local system, they started, but then DPM services would not start.  It turns out that *somehow* the MICROSOFT$DPM$ACCT lost its membership in the local users group... I replaced it, but this did not fix the communication issues.

    Strangely enough, again working on this today, I noticed on the Primary DPM server that If i tried to install an agent onto a box that already had an agent, DPM did not prompt me as per normal about trying the ATTACH function, like it usually does; it just told me that that box already had an agent installed.

    Again, all of these side issues of comms go away when I shutdown the PRI DPM box...

    If you look very closely, at the floor, you can see the PRI DPM box dying a slow and horribly painful death; wriggling and writhing its complaints all the while.

    While we are working around this *issue*; I would like to know what went wrong, and why, but at the cost of $400 bucks or so for a support call,  it is quicker and easier to bypass the problem rather than trying to understand it....

    However, I do have one fear, and that is that all of these issues are related to the fact of all of the SQL databases trying to use  the same account for startup,and maybe DPM lost track of which database to use...

    my secondary DPM server was installed the same way(s), with the same history and at the same time roughly, so I am kinda scared to reboot that box ....

    Rick

    Thursday, September 02, 2010 4:44 AM
  • Not sure why you marked that as an answer.

    Anyway, how do firewall exceptions affect the communication when the firewall is turned off???

    Furthermore, we already have a incoming FW rule that allows ALL communication originating at the DPM Server, so I don't the point in adding those rules (more restrictive) you mention (plus DPM agent installer already created those).

    Also, as I've stated in my original post, firewall rules are enforced by Group Policy settings (and local firewall rules are ignored), so they are the same on all machines. Why would it then work on most machines, but not on some?

    Tuesday, September 07, 2010 1:56 PM
  • Have your tried to run the CMDLET SetDpmServer on the server that's not working?
    Check out my DPM blog @ http://robertanddpm.blogspot.com
    • Proposed as answer by Cornelius N Monday, December 30, 2013 12:26 AM
    Tuesday, September 07, 2010 9:20 PM
    Moderator
  • Robert - not sure how to do that. Do I need to import some module into PowerShell on the client machine?

    Wednesday, September 08, 2010 7:17 AM
  • If you have installed the agent on the server you want to protect you will find the binary at the BIN catalouge in the DPM agenet installtion.

    %system drive%\Program Files\Microsoft Data Protection Manager\DPM\bin

    Run the CMDLET called SetDpmServer with the syntax SetDpmServer -DpmServerName dpmservername

     

    BR

    Robert Hedblom


    Check out my DPM blog @ http://robertanddpm.blogspot.com
    Wednesday, September 08, 2010 8:27 AM
    Moderator
  • Ah, you mean the binary. I've already ran that - see my second post from top above (it returns "SetDpmServer failed with errorcode =0x80004005, error says: Unspecified error").
    Wednesday, September 08, 2010 8:42 AM
  • Markos,

    By defualt the agent installation piece expects certain firewall exceptions to be in place regardless if the firewall service is running or not. Typically these exceptions revolve around the built-in WMI exceptions. If these exceptions are missing then we'll either have agent installation/configuration issues or communication problems. I am not on the team that supports the firewall but I do know that event when not in use there are some instances where it comes into play.

    In most cases with Windows Server 2008 and 2008 R2 we have the needed exceptions in place. However due to build process or role/feature additions or removals the needed exceptions are not in place. So part of troubleshooting the software firewall problems is we check for the default exceptions.

    If there are external firewalls, ISA/TMG and the like, and we have VPN tunnels between DPM and protected servers, there are other things that prevent traffic between the two. Even when allow all rules are in place some DPM RPC/DCOM traffic is dropped. So there are things we have to configure in that case.

    One more thing to take into account is that not all protected server interactios with the DPM server are going to be originated on the DPM side. We'll have DPMRA sessions originating from the protected server in response to DPM intitiated traffic. So we need to allow all in both directions.

    To your question about why, if all servers get the same firewall settings via GPOs, do some not work, my first step is the firewall exceptions. Then go from there.

    /Steve


    Steve L [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by MarkosP Thursday, September 09, 2010 6:09 AM
    Wednesday, September 08, 2010 9:55 AM
    Moderator
  • You said that your firewall setting is distributed by a GPO for all your servers. Those servers that work does they have the samt GPO as the one who doesn't work?


    Check out my DPM blog @ http://robertanddpm.blogspot.com
    Wednesday, September 08, 2010 11:10 AM
    Moderator
  • Thank you Steve for the clarification and sorry for being skeptical before trying the outlined steps above, but I believe the explanation was necessary.

    I've followed the steps outlined in your previous post and the problem was with the 2nd set of builtin rules (the group identified as group=\"@FirewallAPI.dll,-34251\") - those were missing. I've recreated them with the registry entries, then I could attach the agent successfully.

    Still have few questions though:

    1) what are the (friendly) rules names/groups identified by the group=\"@FirewallAPI.dll,-29502\" and group=\"@FirewallAPI.dll,-34251\" in the above netsh commands?

    2) I'm not sure I understand how missing exceptions can affect DPM Agent when the firewall is disabled or turned off completely (the service). So actually the DPM Agent (or the installer) depends on these built-in rules even in the case where the built-in Windows firewall is not used at all? That seems like a very strange design decision.

    3) [quote]One more thing to take into account is that not all protected server interactios with the DPM server are going to be originated on the DPM side. We'll have DPMRA sessions originating from the protected server in response to DPM intitiated traffic. So we need to allow all in both directions[/quote]

    So when we use only the built-in WFAS (which by default doesn't block any outgoing traffic) then we don't have to configure any extra rules, right? (assuming all other communication is transparent, ie. no VPN, no other firewalls between protected servers and DPM server etc.).

    Edit: Robert, you've posted while I was writing this answer, so I've already resolved the problem, but yes, same GPO is applied too all the servers (those that were working and the one that was not).

    Wednesday, September 08, 2010 11:23 AM
  • Markos,

    Glad that cleared some things up for you. Here are the answers for you.

    1. The 'FirewallAPI.dll' ones map to the following. There is some variance between 2008 and 2008 R2.

    Remote Service Management @FirewallAPI.dll,-29502
    This sets the firewall exceptions to all remote (inbound) interaction with Service Control Manager (SCM) over TCP, Named Pipes, and RPC.
    RemoteSvcAdmin-In-TCP
    RemoteSvcAdmin-NP-In-TCP
    RemoteSvcAdmin-RPCSS-In-TCP

    Windows Management Instrumentation @FirewallAPI.dll,-34251
    This sets to allow TCP for DCOM over RPC (in), remote WMI (in), remote WMI (out), and asynchronous WMI (in).
    WMI-RPCSS-In-TCP - Windows Management Instrumentation (DCOM-In)
    WMI-WINMGMT-In-TCP - Windows Management Instrumentation (WMI-In)
    WMI-WINMGMT-Out-TCP - Windows Management Instrumentation (WMI-In)
    WMI-ASYNC-In-TCP - Windows Management Instrumentation (ASync-In)

    2. These exceptions should be on by default. The agent install (or setdpmserver) checks for the exceptions so it may enable the ones needed. This is done regardless if the firewall service is enabled so that if off, when turned on, the needed rules are there. We can't assume that the firewall service state (on/off) will be the state forever so we just prepare. This is not an issue until an out-of-the-box default is missing.

    3. When using built in firewalls we don't have any extra configuration needed after the agent is installed and configured on the protected server. The DPMRA service has an exception set for inbound traffic on any ports it uses on both ends so this plus the WMI and remote SCM settings let us work.

    I hope his answered you questions.

    /Steve


    Steve L [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, September 08, 2010 2:44 PM
    Moderator
  • Thank you Steve, I appreciate the explanation, that answered all questions I had on this topic :)
    Thursday, September 09, 2010 6:08 AM
  • Robert,

    Your suggestion works find for my case, Thank You =]

    Friday, April 27, 2012 7:28 AM