none
DPM 2012 protecting SQL 2008 R2 Cluster (2 nodes) - Error ID 104 Unknown error during Replica creation RRS feed

  • Question

  • Hi,

    We have just installed a SQL 2008 R2 cluster and we trying to protect it with DPM 2012. When I add the cluster to a new protection group I am able to add it with no issues. The replica creation however fails every time. The SQL cluster runs on Windows 2008 R2 SP1. The SQL cluster has a single instance with a single DATA/LOG LUN and one Quorum disk. I also have a disk for the DTC.

    In the DPM console I see this error when creating a replica:

    An unexpected error occurred while the job was running. (ID 104 Details: Unknown error (0x80040e37) (0x80040E37))

    I have made sure that VSS is running and is registered on both cluster nodes. I also made sure that the account it runs as (local system) has sysadmin rights on the cluster instance.

    The following is a part of the DPM error log from the active cluster node:

    07EC	035C	04/16	13:37:55.785	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.785	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.832	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.832	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.832	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.832	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.863	03	localmachinename.cpp(134)			WARNING	GetDnsDomainSuffix failed. wszDnsDomainName = doc.local
    07EC	035C	04/16	13:37:55.863	03	localmachinename.cpp(134)			WARNING	GetDnsDomainSuffix failed. wszDnsDomainName = doc.local
    07EC	035C	04/16	13:37:55.863	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.863	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.879	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.879	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.879	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.879	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.894	31	sqlwriterhelperplugin.cpp(1465)	[000000000039C3E0]		NORMAL	Executing SQLQuery SELECT name FROM sys.master_files WHERE physical_name = N'H:\MSSQL10_50.SQL08R2CLUSTER\MSSQL\DATA\SRV00285-A.mdf'
    07EC	035C	04/16	13:37:55.894	31	sqlwriterhelperplugin.cpp(1465)	[000000000039C3E0]		NORMAL	Executing SQLQuery SELECT name FROM sys.master_files WHERE physical_name = N'H:\MSSQL10_50.SQL08R2CLUSTER\MSSQL\DATA\SRV00285-A_log.LDF'
    07EC	035C	04/16	13:37:55.894	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.894	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.894	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.894	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.894	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.910	31	vssfilereadhelper.cpp(897)			NORMAL	ssLocalVolumeGuid = [\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\], ssClusterVolGuid=[\\?\Volume{a7460043-5f1f-11e1-815b-180373ef8af9}\]
    07EC	035C	04/16	13:37:55.910	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.910	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.910	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type Volume Manager Disk Group with windows error 5078
    07EC	035C	04/16	13:37:55.910	03	clusterutil.cpp(1760)			WARNING	ClusterResourceTypeOpenEnum failed for resource type GeoCluster Replicated Disk with windows error 5078
    07EC	035C	04/16	13:37:55.925	31	sqlwriterhelperplugin.cpp(1126)	[000000000039C3E0]		NORMAL	get Database Recovery model Query = SELECT RECOVERY_MODEL_DESC FROM SYS.DATABASES WHERE NAME = N'SRV00285-A' for instance = SQL08R2CLUSTER
    07EC	035C	04/16	13:37:55.925	03	sqlqueryhelper.cpp(288)	[00000000033CED30]		WARNING	SQL - ICommandText::Execute of "SELECT RECOVERY_MODEL_DESC FROM SYS.DATABASES WHERE NAME = N'SRV00285-A'" - Failed 
    07EC	035C	04/16	13:37:55.925	03	sqlqueryhelper.cpp(332)	[00000000033CED30]		WARNING	SQL - Error Description "Invalid object name 'SYS.DATABASES'."
    07EC	035C	04/16	13:37:55.925	03	sqlqueryhelper.cpp(345)	[00000000033CED30]		WARNING	Failed: Hr: = [0x80040e37] SQL - sql code = 208 (Errorlevel - 0)
    07EC	035C	04/16	13:37:55.925	03	sqlqueryhelper.cpp(353)	[00000000033CED30]		WARNING	SQL - Detailed Description "Invalid object name 'SYS.DATABASES'."
    07EC	035C	04/16	13:37:55.925	31	sqlwriterhelperplugin.cpp(1133)	[000000000039C3E0]		WARNING	Failed: Hr: = [0x80040e37] Error while querying the Database SRV00285-A recovery model from SQLInstance SQL08R2CLUSTER
    07EC	035C	04/16	13:37:55.925	31	sqlwriterhelperplugin.cpp(1135)	[000000000039C3E0]		WARNING	SQL Error Information: Invalid object name 'SYS.DATABASES'.
    07EC	035C	04/16	13:37:55.925	31	aasubtask.cpp(906)	[00000000011793A0]		WARNING	<?xml version="1.0"?>
    07EC	035C	04/16	13:37:55.925	31	aasubtask.cpp(906)	[00000000011793A0]		WARNING	<Status xmlns="http://schemas.microsoft.com/2003/dls/StatusMessages.xsd" StatusCode="-2147217865" Reason="Error" CommandID="RAPreBackup" CommandInstanceID="64c55a82-5398-458e-a7b4-f35ec3281d9a" GuidWorkItem="38dcabbe-6e66-486f-af5b-3dde7f97fa21" TETaskInstanceID="64da0d3b-0c04-4871-9ee6-19adb777165b"><ErrorInfo xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd" ErrorCode="998" DetailedCode="-2147217865" DetailedSource="2"/></Status>
    07EC	035C	04/16	13:37:55.925	03	runtime.cpp(1389)	[000000000046C990]		FATAL	Subtask failure, sending status response XML=[<?xml version="1.0"?>
    07EC	035C	04/16	13:37:55.925	03	runtime.cpp(1389)	[000000000046C990]		FATAL	<Status xmlns="http://schemas.microsoft.com/2003/dls/StatusMessages.xsd" StatusCode="-2147217865" Reason="Error" CommandID="RAPreBackup" CommandInstanceID="64c55a82-5398-458e-a7b4-f35ec3281d9a" GuidWorkItem="38dcabbe-6e66-486f-af5b-3dde7f97fa21" TETaskInstanceID="64da0d3b-0c04-4871-9ee6-19adb777165b"><ErrorInfo xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd" ErrorCode="998" DetailedCode="-2147217865" DetailedSource="2"/></Status>
    07EC	035C	04/16	13:37:55.925	03	runtime.cpp(1389)	[000000000046C990]		FATAL	]

    All firewalls have been disabled between DPM and both SQL nodes. They are on the same subnet.

    Please let me know if you need any more info.

    Thanks,

    Andreas

    Monday, April 16, 2012 2:34 PM

Answers

  • Hello,

    This is a code defect on the SQLWriterHelperPlugin.dll that is running on the Protected Server.<o:p></o:p>

    A simple test to do to confirm that is to run the query:<o:p></o:p>

    SELECT RECOVERY_MODEL_DESC FROM SYS.DATABASES WHERE NAME = N'<name of the database> on your SQL server. It should fail.<o:p></o:p>

    If now you ran the query:<o:p></o:p>

    SELECT RECOVERY_MODEL_DESC FROM sys.databases WHERE NAME = N'<name of the database> . It should work. SYS.DATABASES got
    changed to lower case in this query.<o:p></o:p>

    We are currently testing a private hotfix with premier customer and it's solve the issue. Please open a support incident withMicrosoft Support.<o:p></o:p>

    Thanks<o:p></o:p>

    Carlos Carrolo

    MSFT


    Carlos Carrolo

    • Proposed as answer by rafalwo Tuesday, May 1, 2012 7:53 PM
    • Marked as answer by Andreas1987 Tuesday, May 1, 2012 8:52 PM
    Tuesday, May 1, 2012 6:21 PM
  • 2711168 You receive "Unknown error (0x80040e37)" in System Center Data Protection Manager 2012 when you try to create recovery points for SQL Server or SharePoint databases

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2711168


    Thanks, Wilson Souza - MSFT This posting is provided "AS IS" with no warranties, and confers no rights

    Thursday, May 24, 2012 7:24 AM
    Moderator

All replies

  • Hi Andreas

    Have you solved this yet?

    Br.

    Janick


    Thursday, April 19, 2012 1:30 PM
  • No i have no idea what could be causing it or how to fix it. All other backups work perfectly fine. I am also backing up regular SQL servers and a Hyperv Cluster. The problem only exists with the SQL cluster (which I only have one).

    Let me know if there is anything I can try.

    Thanks,

    Andreas

    Thursday, April 19, 2012 1:33 PM
  • Same for us.

    I will return with info if we succesfully solves this issue.

    /Janick

    Thursday, April 19, 2012 1:40 PM
  • Hey Guys,

    i got the same issue with some SQL 2005 Servers. Absolute the same behaivor. I also raised a ticket at MS as a premier Partner with the info that SQL Backup is still a BUG and we/i need to wait for the next QFE

    Friday, April 27, 2012 9:53 AM
  • Thanks for the info! Do you know when the QFE will be released?

    Thanks,

    Andreas

    Friday, April 27, 2012 12:19 PM
  • What type of collation you have set?
    Is it case sensitive?

    In this log you have line:

    Recovery model Query = SELECT RECOVERY_MODEL_DESC FROM SYS.DATABASES WHERE NAME = N'SRV00285

    I have a similar problem only with databases, when I was set case sensitive collation.

    Is it any way to change size of chars in this DPM query?

    Friday, April 27, 2012 12:29 PM
  • Yes the collation of that server is set to Case sensitive. It happens to be the only server I am backing up with a default collation of Case sensitive.

    Looks like it might be a collation problem.

    Is this the QFE Jan was talking about?

    Thanks,
    Andreas

    Monday, April 30, 2012 1:54 PM
  • had a install with the same issue today , opening a case with premier tomorrow will update when i hear something

    (also with case sentesive collation)
    Monday, April 30, 2012 5:46 PM
  • Hello,

    This is a code defect on the SQLWriterHelperPlugin.dll that is running on the Protected Server.<o:p></o:p>

    A simple test to do to confirm that is to run the query:<o:p></o:p>

    SELECT RECOVERY_MODEL_DESC FROM SYS.DATABASES WHERE NAME = N'<name of the database> on your SQL server. It should fail.<o:p></o:p>

    If now you ran the query:<o:p></o:p>

    SELECT RECOVERY_MODEL_DESC FROM sys.databases WHERE NAME = N'<name of the database> . It should work. SYS.DATABASES got
    changed to lower case in this query.<o:p></o:p>

    We are currently testing a private hotfix with premier customer and it's solve the issue. Please open a support incident withMicrosoft Support.<o:p></o:p>

    Thanks<o:p></o:p>

    Carlos Carrolo

    MSFT


    Carlos Carrolo

    • Proposed as answer by rafalwo Tuesday, May 1, 2012 7:53 PM
    • Marked as answer by Andreas1987 Tuesday, May 1, 2012 8:52 PM
    Tuesday, May 1, 2012 6:21 PM
  • How do I open support incident? Could you help?

    This hot fix is very important for me because I temporary use different way to backup MsSql.

    and would like to use the DPM functionality such as backup every 15 minutes

    Best regards

    Rafal

    Tuesday, May 1, 2012 7:55 PM
  • I just received the private hotfix and it works. In order to request support you can go through this website:

    https://support.microsoft.com/oas/default.aspx?gprid=16340&st=1&wfxredirect=1&sd=gn

    Professional support is not free unless you have a silver/gold partner agreement or software assurance. (there is also other ways for you to have free support cases) They do require that you use a support incident for it initially however they refund the support incident once they confirm that this is a bug. I simply referenced this post and they did the check with the query  like Carlos suggested and they will give you the hotfix and refund the support incident you used to open the request.

    Thanks,

    Andreas

    Tuesday, May 1, 2012 8:52 PM
  • Thanks Carlos for the quick help today :)

    Installed the updated dll on 2 servers so far and both are running , restore test i scheduled for tomorrow

    Tuesday, May 1, 2012 9:23 PM
    Moderator
  • Looks like the PSS site wont process any cases.. :(    Any idea on how soon this fix will be generally available ?  I dont really have time to sit on the phone....
    Wednesday, May 2, 2012 8:27 PM
  • I was told it might be in the next Update Rollup however I don't know when that will be.

    Try this link with IE:

    https://support.microsoft.com/oas/default.aspx?gprid=16340&st=1&wfxredirect=1&sd=gn

    Firefox won't open it. Once you submit the case and they call you it takes all of 20 minutes until you have the fix.

    Good luck.

    Andreas

    Wednesday, May 2, 2012 8:31 PM
  • Managed to fight though PSS.. sheesh.. looks like the updated SQL vss writer .dll fixes the issue.

    Friday, May 4, 2012 2:01 PM
  • 2711168 You receive "Unknown error (0x80040e37)" in System Center Data Protection Manager 2012 when you try to create recovery points for SQL Server or SharePoint databases

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2711168


    Thanks, Wilson Souza - MSFT This posting is provided "AS IS" with no warranties, and confers no rights

    Thursday, May 24, 2012 7:24 AM
    Moderator