locked
DFSR - Constant 1726 Events and large files do not replicate. RRS feed

  • Question

  • Hi,

    We have a DFSR hub and spoke topology with 50 members across 45 sites connected with a mixture of IPSEC VPN & MPLS. 1 Hub server running Server 2012 Standard, all others are either Server 2008 R2 Standard or Enterprise.

    All servers have the latest patchlevel
    All servers are in one domain
    Theres no problem with normal AD-Replication
    No obvious errors from dcdiag or netdiag
    1 Namespace
    61 Replicated Folders containing a mixture of office documents, images and videos

    Problem:
    There are large backlogs reported by DFSR diag
    Large files replicate extremely slowly or not at all (for example a 984MB file created 32 days ago has still not reached 40 out of 50 sites.)
    Many DFSR Event ID 5014 warnings are logged on all members ('Error: 1726 (The remote procedure call failed.) 
    The hub server's DFSR debug logs contain many entries stating 'Failed to reserve file transfer server context, no more contexts are available'. These messages seem to corrospond with the DFSR event 5014 messages.

    Example debug log:

    20131201 18:03:46.908 3696 SRTR  2772 InitializeFileTransferAsyncState::ProcessIoCompletion 0000003401A87920
    20131201 18:03:46.908 3696 RPCN  1889 [WARN] NetworkGlobals::ReserveFileTransfer Failed to reserve file transfer server context, no more contexts available. updateName:removedfilename.ods uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665723 gvsn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665723 connId:{1F655F11-0101-4833-805E-680361C778FE} csId:{ACF6E46C-09CE-4E7D-85F5-10206356C231} totalContexts:128
    20131201 18:03:46.908 3696 SRTR  3011 [WARN] InitializeFileTransferAsyncState::ProcessIoCompletion Failed to initialize a file transfer. connId:{1F655F11-0101-4833-805E-680361C778FE} rdc:1 uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665723 gsvn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665723 completion:0 ptr:0000003401A87920 Error:[Error:9078(0x2376) InitializeFileTransferAsyncState::ProcessIoCompletion servertransport.cpp:2884 3696 C All server file transfer contexts are currently busy] 
    20131201 18:03:46.908 3696 SRTR  2701 InitializeFileTransferAsyncState::~InitializeFileTransferAsyncState this:0000003401A87920
    20131201 18:03:46.908 3696 SRTR  2772 InitializeFileTransferAsyncState::ProcessIoCompletion 0000003401A85D00
    20131201 18:03:46.908 3696 RPCN  1889 [WARN] NetworkGlobals::ReserveFileTransfer Failed to reserve file transfer server context, no more contexts available. updateName:removedfilename.xls uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1608783 gvsn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1608783 connId:{1F655F11-0101-4833-805E-680361C778FE} csId:{ACF6E46C-09CE-4E7D-85F5-10206356C231} totalContexts:128
    20131201 18:03:46.908 3696 SRTR  3011 [WARN] InitializeFileTransferAsyncState::ProcessIoCompletion Failed to initialize a file transfer. connId:{1F655F11-0101-4833-805E-680361C778FE} rdc:1 uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1608783 gsvn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1608783 completion:0 ptr:0000003401A85D00 Error:[Error:9078(0x2376) InitializeFileTransferAsyncState::ProcessIoCompletion servertransport.cpp:2884 3696 C All server file transfer contexts are currently busy] 
    20131201 18:03:46.908 3696 SRTR  2701 InitializeFileTransferAsyncState::~InitializeFileTransferAsyncState this:0000003401A85D00
    20131201 18:03:46.908 3696 SRTR  2772 InitializeFileTransferAsyncState::ProcessIoCompletion 0000003401A87530
    20131201 18:03:46.908 3696 RPCN  1889 [WARN] NetworkGlobals::ReserveFileTransfer Failed to reserve file transfer server context, no more contexts available. updateName:removedfilename.pdf uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665542 gvsn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665542 connId:{1F655F11-0101-4833-805E-680361C778FE} csId:{ACF6E46C-09CE-4E7D-85F5-10206356C231} totalContexts:128
    20131201 18:03:46.908 3696 SRTR  3011 [WARN] InitializeFileTransferAsyncState::ProcessIoCompletion Failed to initialize a file transfer. connId:{1F655F11-0101-4833-805E-680361C778FE} rdc:1 uid:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665542 gsvn:{80EBA14C-E68B-438F-AC8F-22D9B32BF51D}-v1665542 completion:0 ptr:0000003401A87530 Error:[Error:9078(0x2376) InitializeFileTransferAsyncState::ProcessIoCompletion servertransport.cpp:2884 3696 C All server file transfer contexts are currently busy] 
    20131201 18:03:46.908 3696 SRTR  2701 InitializeFileTransferAsyncState::~InitializeFileTransferAsyncState this:0000003401A87530
    20131201 18:03:46.908 3696 SRTR  2772 InitializeFileTransferAsyncState::ProcessIoCompletion 0000003401A83510
    20131201 18:03:46.908 3696 RPCN  1889 [WARN] NetworkGlobals::ReserveFileTransfer Failed to reserve file transfer server context, no more contexts available. updateName:removedfilename.doc uid:{1DA51289-C37E-4A07-A625-90933A0F808D}-v1412159 gvsn:{1DA51289-C37E-4A07-A625-90933A0F808D}-v1412159 connId:{4344F679-9507-4D15-9730-DFC73EFAD456} csId:{ACF6E46C-09CE-4E7D-85F5-10206356C231} totalContexts:128
    20131201 18:03:46.924 3696 SRTR  3011 [WARN] InitializeFileTransferAsyncState::ProcessIoCompletion Failed to initialize a file transfer. connId:{4344F679-9507-4D15-9730-DFC73EFAD456} rdc:1 uid:{1DA51289-C37E-4A07-A625-90933A0F808D}-v1412159 gsvn:{1DA51289-C37E-4A07-A625-90933A0F808D}-v1412159 completion:0 ptr:0000003401A83510 Error:[Error:9078(0x2376) InitializeFileTransferAsyncState::ProcessIoCompletion servertransport.cpp:2884 3696 C All server file transfer contexts are currently busy] 
    20131201 18:03:46.924 3696 SRTR  2701 InitializeFileTransferAsyncState::~InitializeFileTransferAsyncState this:0000003401A83510
    20131201 18:03:46.924 3696 SRTR  2772 InitializeFileTransferAsyncState::ProcessIoCompletion 0000003401A864E0

    I have tried the following solutions:

    Installed latest NIC-drivers on each server
    Installed latest DFSR binaries
    No teaming at the NICs
    Disabled task offloading 
    Increased the size of the staging folder to 16GB
    Excluded DFSRs.exe from Antivirus
    Excluded 'System Volume Information' directory from Antivirus
    No backups are running
    Increased Bandwidth allocation
    Tried RDC disabled & Enabled - no difference
    Removed any FSRM screens
    All members are pre-staged using the Robocopy method
    reviewed http://blogs.technet.com/b/askds/archive/2007/10/05/top-10-common-causes-of-slow-replication-with-dfsr.aspx
    reviewed http://support.microsoft.com/default.aspx?scid=kb%3bEN-US%3b958802 to get the latest DFS-Files

    Any suggestions please?


    • Edited by iand4403 Sunday, December 1, 2013 6:39 PM formatting
    Sunday, December 1, 2013 6:36 PM