none
Problem with MSDTC

    Question

  • Ive been working on this error for the past week without much success.

    Context : I have a bunch of server running Sql server 2005 under Windows 2003(enterprise,standard and x64) and 2 new servers running Sql server 2008 under Windows 2008 Standard. I use MS DTC with sql server and clr assemblies to transfer and modify some data to a datawarehouse.

    Our 2 new windows 2008 servers have the same hardware configuration and basic software configuration. One of them wich we will call gisServer is running ArcGis and Sql Server. The other one wich we will call prodServer is running Sql server only.

    Problem : On my prodServer MSDTC is not working properly but on the other one its fine.

    Heres what ive done with what im thinking in order of date starting When i discovered the problem.

    1. Im getting a bunch of msdtc errors under sql server. SSMS returns the following error : MSDTC on server 'prodServer' is unavailable.
    2. In the event viewer I always get the 2 same entries when i try a transaction :
    • Attempting to initialize Microsoft Distributed Transaction Coordinator (MS DTC). This is an informational message only. No user action is required.
    • The Microsoft Distributed Transaction Coordinator (MS DTC) service could not be contacted.  If you would like distributed transaction functionality, please start this service.
    3. I checked in the component service and MSDTC is started. I also checked his dependency : COM+ Event System (optional) , COM+ System Application ,                                             DCOM Server Process Launcher , Distributed Transaction Coordinator , Message Queuing , and Remote Procedure Call (RPC) are also started.
    4. I checked with the network analyst and we only have a firewall for requests going to Internet. In our internal network there are no firewall .
    5. I also checked if there are network policy and if windows firewall was started. No network policy and no windows firewall enabled.
    6. Current MSDTC configuration
    • Network DTC Access : on
    • Allow Remote Clients : on
    • Allow Remote Administration : on
    • Allow Inbound : on
    • Allow Outbout : on
    • No Authentification Required
    • Enable XA Transactions : on
    • Logon Account : NT AUTHORITY\NetworkService
    7. Tried flusing the logs and traces.
    8. Tried reinstalling msdtc. No difference.
    9. I tried enabling msdtc via Server Manager by installing Application Role(wich provide support for msdtc)
    10. Tried dtctester here's the output


    MSTDC on server is unavailable
    • Firewall has ports closed
    • bad WINS/DNS
    • Misconfigured network
    • Misconfigured SQL Server machine that has multiple netcards.

    12. I have enabled Netbios over tcp ip.
    13. I Noticed MSDTC service stopped by itself during troubleshooting and i can't find any information on what caused it.
    14. I got few interesting log entry :
    • MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection with system prodServer.
    • Invalid command line arguments. (for MSDTC in the logs)
    (i havent tried to start MSDTC from command line and it only does it from time to time...
    15. MS DTC Tracing infrastructure : the attempt to flush the existing trace data failed. Internal Information : none available.
    16. I RE RE RE RE RE RECHECKED that the service and its dependency were started.
    17. Made a clr trigger on sql server to see if i can cause more interesting errors that could lead me to understanding the problem. here's what im getting(wich is no more interesing)
    • System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.Transactions.TransactionPromotionException: MSDTC on server 'prodServer' is unavailable. ---> System.Data.SqlClient.SqlException: MSDTC on server 'prodServer' is unavailable.
    18. I have checked about every help on internet and still can't find the answer.
    19. In MSDTC traces i only see that transaction get aborded but no more information.
    20. In component service, msdtc statistics are always at 0 even though i try generating transactions. It dosen't event registers them as aborded(but the traces does ???)
    21. I also checked with process explorer and i can see msdtc running and i see its footprint going up when i generate trasactions.
    22. Using debugging logs i get these events :
    • The WS-AT protocol service successfully completed startup and recovery.
    •  Protocol ID: c05b9cad-ab24-4bb3-9440-3548fa7b4b1b
    •  Protocol Name: WS-AtomicTransaction 1.1
    •  Process Name: msdtc
    •  Process ID: 9204

    HERES THE CONFIGURATION :
    Microsoft Windows [Version 6.0.6001]

    Host Name:                 prodServer
    OS Name:                   Microsoft® Windows Server® 2008 Standard without Hyper-V
    OS Version:                6.0.6001 Service Pack 1 Build 6001
    OS Manufacturer:           Microsoft Corporation
    OS Configuration:          Member Server
    OS Build Type:             Multiprocessor Free
    Registered Owner:          Windows User
    Registered Organization:
    Product ID:                ********************
    Original Install Date:     03/03/2009, 9:44:46 PM
    System Boot Time:          25/06/2009, 9:12:22 AM
    System Manufacturer:       HP
    System Model:              ProLiant DL380 G5
    System Type:               x64-based PC
    Processor(s):              2 Processor(s) Installed.
                               [01]: Intel64 Family 6 Model 23 Stepping 10 GenuineIn
    tel ~2833 Mhz
                               [02]: Intel64 Family 6 Model 23 Stepping 10 GenuineIn
    tel ~2833 Mhz
    BIOS Version:              *****
    Windows Directory:         C:\Windows
    System Directory:          C:\Windows\system32
    Boot Device:               \Device\HarddiskVolume1
    System Locale:             en-ca;English (Canada)
    Input Locale:              en-us;English (United States)
    Time Zone:                 (GMT-05:00) Eastern Time (US & Canada)
    Total Physical Memory:     12,285 MB
    Available Physical Memory: 427 MB
    Page File: Max Size:       24,781 MB
    Page File: Available:      13,463 MB
    Page File: In Use:         11,318 MB
    Page File Location(s):     C:\pagefile.sys
    Domain:                    ********
    Logon Server:              \\******
    Hotfix(s):                 44 Hotfix(s) Installed.
                               [01]: {47740627-D81D-4A45-A215-03B075A18EC7}
                               [02]: {5F7F6FFF-395D-480E-8450-64F385D82C5F}
                               [03]: {5E10E16E-CF65-479B-8E64-E240E621F5D2}
                               [04]: {B0D673F4-01F4-44E6-8944-4F22A558D042}
                               [05]: {F9852BE7-7A55-413C-BBD0-C9B72790E983}
                               [06]: {012BE245-FE02-466B-9835-209FE4FB7698}
                               [07]: {65AFCB03-3111-47E2-B1ED-198D6B0BC96E}
                               [08]: KB940518
                               [09]: KB938464
                               [10]: KB942288
                               [11]: KB948609
                               [12]: KB948610
                               [13]: KB949189
                               [14]: KB950050
                               [15]: KB950760
                               [16]: KB950762
                               [17]: KB950974
                               [18]: KB951066
                               [19]: KB951698
                               [20]: KB951978
                               [21]: KB952287
                               [22]: KB953733
                               [23]: KB954211
                               [24]: KB954459
                               [25]: KB955020
                               [26]: KB955069
                               [27]: KB955302
                               [28]: KB955839
                               [29]: KB956802
                               [30]: KB956841
                               [31]: KB957097
                               [32]: KB957200
                               [33]: KB957321
                               [34]: KB957388
                               [35]: KB958481
                               [36]: KB958483
                               [37]: KB958623
                               [38]: KB958624
                               [39]: KB958644
                               [40]: KB958687
                               [41]: KB959130
                               [42]: KB960715
                               [43]: KB961260
                               [44]: KB967190
    Network Card(s):           4 NIC(s) Installed.
                               [01]: HP NC373T PCIe Multifunction Gig Server Adapter

                                     Connection Name: Local Area Connection
                                     DHCP Enabled:    No
                                     IP address(es)
                                     [01]: **********
                                     [02]: *****************
                               [02]: HP NC373T PCIe Multifunction Gig Server Adapter

                                     Connection Name: Local Area Connection 2
                                     DHCP Enabled:    No
                                     IP address(es)
                                     [01]: **********
                                     [02]: ********************
                               [03]: HP NC373i Multifunction Gigabit Server Adapter
                                     Connection Name: Local Area Connection 3
                                     Status:          Media disconnected
                               [04]: HP NC373i Multifunction Gigabit Server Adapter
                                     Connection Name: Local Area Connection 4
                                     DHCP Enabled:    No
                                     IP address(es)
                                     [01]: ***********

    I blanked out some information for security but you should have every important information

    NOTE : This is not a problem with SQL SERVER my problem is with MSDTC. I cannot use any msdtc ressource (either with sql server or without).

    Regards
    Gabriel Gb
    • Edited by Gabrielgb Thursday, August 27, 2009 7:28 PM corections
    Thursday, August 27, 2009 6:05 PM

Answers

All replies

  • Hi,

     

    This issue can occur if there’s not enough permission for “Network Service” account for MSDTC service.

     

    Please try steps below to troubleshoot t the issue:

     

    Step 1: Download a tool called SUBINACL from the link below:

    http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B&displaylang=en

     

    Step 2: Open Command Prompt in elevated mode, change the directory to the folder containing subinacl.exe and then run the following command:

    subinacl /service msdtc /grant="Network Service"=QSETIL

     

    The meaning of the letters (can be found as well in the subinacl.htm file in the subinacl’s installation folder):

     

    Q : Query Service Configuration

    S : Query Service Status

    E : Enumerate Dependent Services

    T : Start Service

    I : Interrogate Service

    L : Read Control

     

    For more information, please refer to the following article:

     

    Managing Accounts and Privileges

    http://technet.microsoft.com/en-us/library/cc770953(WS.10).aspx

     

    Enable Network Access Securely for MS DTC

    http://technet.microsoft.com/en-us/library/cc753620(WS.10).aspx

     

    Hope it helps.

     

    Tim Quan - MSFT

    Friday, August 28, 2009 6:49 AM
    Moderator
  • Hi

     

    How are things going? I have not heard back from you in a few days and wanted to check on the status of the issue. Please let me know how things turned out.

     

    Tim Quan - MSFT

    Monday, August 31, 2009 2:28 AM
    Moderator
  • Just had this problem with our system test environment. The environment has been working for 12+ months (with MSDTC configured for distributed transactions), then starting getting the MSDTC on server 'server' is unavailable error a couple of days ago. DTCPing showed that MSDTC was working OK. SQL Server appeared to be working OK also. Tried the fix above, but that didn't work. Was just about to rebuild the DB server when I noticed a log file for one of the databases was exceptionally large (5.7Gb, DB is only 100Mb).

    Truncated the log file and everything started working again.

    Best guess is the log file was corrupted and this was somehow preventing SQL Server from rolling back and/or checking for aborted distributed transaction. There were no SQL Server Log or Event Log entries that I could find to indicate a problem with the database or the DB log file. Note that the database with the corrupt log file was not the one used by our application. Also, that DB did not need to participate in any distributed transactions.

    Script to truncate a log file is below -> USE AT YOUR OWN RISK! If you don't know what the DB log file does - TALK TO YOUR DBA FIRST!

    -- Get name of DB

    Select * from master..sysdatabases

     

    BACKUP LOG <DBName> WITH TRUNCATE_ONLY   -- Use DB Name

    DBCC SHRINKFILE(<LogFileName>,10)                         -- Use Logical log name

     

    Tuesday, October 20, 2009 12:29 AM
  • I did a restart and msdtc started working. Look at my steps and make sure everything is right then restart.

    Gabriel Gosselin B.
    DBA
    Altus Group
    Tuesday, October 20, 2009 12:33 AM
  • have you done any recent update ?
    Tuesday, October 20, 2009 2:30 AM
  • Yes - Applied the latest Windows Updates just before the error started to occur so initially thought that one of the updates was the cause. However after uninstalling all the updates (and rebooting) the problem persisted.  Wasn't until the the log file was truncated that the error went away.
    Tuesday, October 20, 2009 3:25 AM
  • Hi Tim,

    I have the same problem with Windows Server 2008 SP2. I downloaded and installed SUBINACL tool but I'm getting OpenService Error : 5 Access is denied while using your command line or just subinacl /service msdtc

    The other serveice work fine (e.g. subinacl /service msmq) and I'm running my cmd as Administrator.

    Please help

    V

    Friday, March 18, 2011 6:14 PM
  • I have same issue MSDTC throws error "MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection to 'localhost'" I tried SUBINCAL also but it is not authorized. I am on 2008R2 with a fresh install only 4 days ago.
    Tuesday, March 29, 2011 6:58 PM
  • Hi,

    I have the same problem:

    msdtc - OpenService Error : 5 Zugriff verweigert

     

    Best,

    Thomas

    Wednesday, March 30, 2011 8:21 AM
  • I have the same problem with Windows Server 2008 SP2. I downloaded and installed SUBINACL tool but I'm getting OpenService Error : 5 Access is denied while using your


    ditto - brand spanking new Windows 2008 R2 installation less than 36 hours old and am hitting this issue (amongst others) - OpenService Error : 5 Access is denied

    **have open support case and will repost later if any pertinent information is discovered**

    help!


    Dale Unroe
    Thursday, May 5, 2011 7:44 PM
  • in my case the new server was joined into an existing Windows 2003 domain of which had a lengthy history and may well have been a Windows 2000 domain.  The importance of this legacy is that there was a time previously in which through an MMC you could apply 'security templates' that could configure large scale changes including through Group Policy.  What I discovered during troubleshooting was that the Default Domain Controller Policy as well as the Default Domain Policy were filled with such seemingly security related lock downs and these were now having adverse consequences never originally intended.  These impacted registry keys and folder permissions.

    Once discovered I had to use a known good domain controller in a separate domain as a standard by which to compare this over secured FUBR'd domain.  I set the two above mentioned GPO's back to match this standard.

    The next step was complex and involved using Sysinternal's Process Monitor.  For each service that wouldn't start I had to have Proc Mon on and logging while the 'access denied' events occurred whether on registry keys, folders, or files.  Each had its own unique permission changes required and so I had to go back to the standard 'known good quantity' system for specific permission setting guidance.

    The MSDTC was actually fixed directly by the Microsoft Support Engineer and involved its inability to write to the log (as I now recall days later).  I will correspond with him to get specific clarification on the specific steps so to close this out and then write them here later.


    Dale Unroe

    Saturday, May 7, 2011 4:47 PM
  • The reply from the engineer:

    "I had added the network service account with full rights to the MSDT log file in C:\windows\system32\msdtc\msdt"


    Dale Unroe
    • Proposed as answer by Larry Song CN Friday, December 8, 2017 4:07 AM
    Tuesday, May 10, 2011 3:16 AM
  • This solution is Working but if You are on a cluster than you have to change the MSDTC name in the Clusterd MSDTC Else you are using the local MSDTC and than it Won't Work.

    Saw the same issue Today and fixed it Did Some testing and it Worked. The Clusterd MSDTC look like something like this

    MSDTC$92d91bb1-e731-4c-1688158

    Look the Services up in the Services and copy paste it

    subinacl /service MSDTC$92d91bb1-e731-4c-1688158 /grant="Network Service"=QSETIL

    And Yes in a Admin CMD do not make the mastake and say I logged on as an admin.  Then you have the access denied.


    Greetings, Robert Smit [MVP] http://robertsmit.wordpress.com/
    Friday, June 10, 2011 12:46 PM
  • I downloaded and run SUBINACL tool as directed above and later restart COM+ services.

     

    It resolved COM+ "MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection with system <system name>".

     

    Thanks.

     

    Friday, August 5, 2011 10:47 AM
  • I used "icacls" on my 2008 R2 servers, and the following (note you must have full access to the \msdtc folder to do this)

    icacls c:\windows\system32\msdtc\msdtc.log /grant "networkservice":F

    The icacls tool is already builtin, nothing to download 


    • Edited by Carol Fuchser-Burns Friday, September 2, 2011 6:41 PM
    • Proposed as answer by zWaR Monday, January 2, 2012 10:06 AM
    Friday, September 2, 2011 6:41 PM
  • I had the same problem, but It was because of the windows updates, then I had to restart the server, but I did not have done it yet. When I got aware of that I restarted the server and then It worked ok. Regards
    Monday, November 14, 2011 7:32 PM
  • I had similar problem with clustered MSDTC. By granting full rights to the networkservice on msdtc.log resolved my issue.

     

    Thanks Carol

    Tuesday, January 31, 2012 2:36 AM
  • I just had the same issue on a clustered MSDTC being in the same clustered service as SQL Server 2008R2. None of the solutions above worked. All storage LUNs were on our Netapp storage, mounted as Volume Mount Points, with SQL running correctly and only DTC failing to start. Finally we sorted it out setting DTC to write on a LUN mounted as a drive letter, instead of being on a Volume Mount Point.
    Tuesday, March 6, 2012 3:43 PM
  • Try to restart Distributed Transaction Coordinator sevice. This should fix the issues.
    Thursday, September 13, 2012 7:36 PM
  • Carol, you were 100% on target !!! Thanks Much.

    The below is the content of an internal email I sent to those on my team, for the experience of leaning what really should be done in conjunction with your most excellent and simple post.  As for me there were other issues as explained below.  For those who want the real scoop on what is going on here, I hope you find this email reply to Carol Fuchser-Burns appropiate and useful.

    Hank Freeman

    Senior Systems, Database/Data Warehouse Architect

    To: All … I have taken the time to detail the resolution of the below, for this may be a keeper email, which I call a Twealk .  As for me it is a Blue Moon Event.

    As you know we ran into a situation with GAATLT445W this morning with the following error:

    Communication with the underlying transaction manager has failed.

    In event viewer(Warning):MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection with system GAATLT445W.

    http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/eb8835d9-5f5b-4df2-8c68-4a61d5e44d6b/

    There were several options available to resolve the issue. While this was going on James was looking at and making change to MSDTC on T444W.  In private emails he advise the below which I have documented just in case the whole team needs to know how to resolve

    this ONCE IN THEIR LIFE TIME (Blue Moon) EVENT…

    What I have done on GAATLT444W and T445W is the following

      • Installed SQL server 2008 r2 SP2 – Which was needed anyway.
      • Installed the fix for the aforementioned link from an Administrator CMD prompt Window.

    icacls c:\windows\system32\msdtc\msdtc.log /grant "networkservice":F

    Microsoft Windows [Version 6.1.7601]

    Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

    C:\Users\hfreeman.adm>icacls c:\windows\system32\msdtc\msdtc.log /grant "networkservice":F

    processed file: c:\windows\system32\msdtc\msdtc.log

    Successfully processed 1 files; Failed processing 0 files

    1. Made the below changes to MSTC as described in detail below.
    • This previously had not been set on these two development/test servers and was clearly needed to resolve the reported problem.

    Apparently on the two server T445W and T444w, the below   was not set on Component Services/Microsoft Distributed Transaction   Coordinator.

    So after being advised from James that he modified T444w,   I also changed T445w by selecting (No Authentication Required) per   James’s request/suggestion.

    • Select Component Services/Computer/My   Computer/Select Distributed Transaction Coordinator/Local DTC à Properties
    • Select the Security tab
    • Change Transaction Manager Communications by   selecting the radio button for:  No Authentication Required.
      • When   you apply this setting the service will stop and restart, so make sure it   does or you will have to do so manually.
    • Note: I have this set to Startup Type (Automatic (Delayed Start) because this   server has I-scsi / MPIO disk subsystems, which was also needed on all the   SQL Server and SQL Server Agent Instances

       Picture #2

    ….

    Hank Freeman

    Senior SQL Server DBA/Data & Systems Architect

    Atlanta, Georgia !

    end..end..


    FHankFreeman

    Thursday, October 4, 2012 6:17 PM
  • Hey friends, I have also same problem getting an error on my passive node "Failed trying to get the state of the cluster node: "MSDTC_Network_Name"The error code returned: 0x80070005" with warning message "MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection with system Node2. When I checked the properties of cluster DTC, I found that security settings are not configured. is it because of that I am getting an error.
    • Proposed as answer by Ritesh Parab Thursday, October 11, 2012 9:45 AM
    Monday, October 8, 2012 1:56 PM
  • After spending half the day trying to make MSDTC work I figured out MSDTC does not work with cloned servers. Here's the error it generates - 

    The local MS DTC detected that the MS DTC on EADB has the same unique identity as the local MS DTC. This means that the two MS DTC will not be able to communicate with each other. This problem typically occurs if one of the systems were cloned using unsupported cloning tools. MS DTC requires that the systems be cloned using supported cloning tools such as SYSPREP. Running 'msdtc -uninstall' and then 'msdtc -install' from the command prompt will fix the problem. Note: Running 'msdtc -uninstall' will result in the system losing all MS DTC configuration information.

    I hope it helps someone.

    • Proposed as answer by arvkoz Thursday, September 22, 2016 6:27 AM
    • Unproposed as answer by arvkoz Thursday, September 22, 2016 6:27 AM
    Thursday, August 1, 2013 10:03 AM
  • I faced same problem, the only resolution that worked with me is:

    Just apply these setting on the Component services, using the Domain Admin User.

    After that every thing works fine.

    • Proposed as answer by Thanker Monday, February 10, 2014 3:09 PM
    Monday, February 10, 2014 3:08 PM
  • This fixed it for me.

    https://www.mlakartechtalk.com/msdtc-troubleshooting/

    Friday, April 5, 2019 2:43 PM