none
ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)

    Question

  • Whateve we got this error on DPM2010
    "ID 2019 Details: An existing connection was forcibly closed by the remote host (0x80072746)"

    We always got this error on target server "Volume Shadow Copy Service error: The shadow copy could not be committed - operation timed out. Error context: DeviceIoControl(\\?\Volume{fc1f0549-9865-4349-b4df-8596a19ad7fe} - 000001B4,0x0053c010,0003D5D0,0,0003E5D8,4096,[0])."

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp."

    How to fix this issue?

    Monday, February 13, 2012 6:40 AM

All replies

  • One possible Resolution is to re-register the Volume Shadow Copy DLL's per what shows below:


    Procedure to re-register the Volume Shadow Copy service DLL's:


    CD to Windows\System32
    Net stop vss
    Net stop swprv

    regsvr32 ole32.dll
    regsvr32 vss_ps.dll

    Vssvc /Register

    regsvr32 /i swprv.dll
    regsvr32 /i eventcls.dll

    regsvr32 es.dll
    regsvr32 stdprov.dll
    regsvr32 vssui.dll
    regsvr32 msxml.dll
    regsvr32 msxml3.dll
    regsvr32 msxml4.dll

    Reboot the server.

    After reboot, open a command prompt and run:
    vssadmin list writers



    A Volume Shadow Copy Service (VSS) update package is available for Windows Server 2003
    http://support.microsoft.com/default.aspx?scid=kb;en-us;940349

    Then read this:

    http://technet.microsoft.com/en-us/library/dd364887(WS.10).aspx

    and

    http://technet.microsoft.com/en-us/library/cc734239(WS.10).aspx

     
    Monday, February 13, 2012 3:52 PM
  • Hello,

    That particular error message can be a false positive in some cases. I do have some questions if I may.

    1. On the protected server are there any VSS or Volsnap related errors in the application event log or system event log?

    2. If during a backup attempt if you go to the target server and from an administrator command prompt type:
        a.) vssadmin list writers    <----do you see any writers that are in a failed or hung state?
        b.) vssadmin list shadows  <----do you see any shadow copies that are actually made?

    3. What type of protection is this? Exchange? Sharepoint? BMR?

    4. Are these files SIS enabled?

    5. This can be caused by a slow connectivity between the DPM server and the target server. Is this link being hammered with traffic?

    Thanks
    Shane

    Wednesday, February 15, 2012 12:32 AM
  • Hi good guessing.

    1. Yes, there are VSS and VolSnap errors and we have numerous of errors message similar like this on event viewer. OS: Windows 2003 R2 (32Bit).

    Servers

    Event ID

    Source

    Description

    FileServer02

    5

    VolSnap

    The shadow copy of volume F: could not be created due to insufficient non-paged memory pool for a bitmap structure.

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    FileServer02

    8

    VolSnap

    The flush and hold writes operation on volume F: timed out while waiting for a release writes command.

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

     FileServer01
    FileServer02

    13

    VolSnap

    The shadow copy of volume F: could not grow its shadow copy storage on volume F:.

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    FileServer01
    FileServer02

    35

    VolSnap

    The shadow copies of volume F: were aborted because the shadow copy storage failed to grow.

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    FileServer02

    12310

    VSS

    Volume Shadow Copy Service error: The shadow copy could not be committed - operation timed out. Error context: DeviceIoControl(\\?\Volume{fc1f0549-9865-4349-b4df-8596a19ad7fe} - 00000200,0x0053c010,0003D358,0,0003E360,4096,[0]).

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    FileServer02

    12298

    VSS

    Volume Shadow Copy Service error: The I/O writes cannot be held during the shadow copy creation period on volume \\?\Volume{fc1f0549-9865-4349-b4df-8596a19ad7fe}\. The volume index in the shadow copy set is 0. Error details: Open[0x00000000], Flush[0x00000000], Release[0x80042314], OnRun[0x00000000].

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.


    2. The error accours on pass midnight. Is hard to do that. However I just do it before posting this message.

    C:\>vssadmin list writers

    vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool

    (C) Copyright 2001 Microsoft Corp.

     

    Writer name: 'System Writer'

       Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}

       Writer Instance Id: {4f712f8e-64e6-4104-b193-ce8798464222}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'FSRM Writer'

       Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}

       Writer Instance Id: {afabb380-ae0c-4e68-840c-2a6b66a7d9a4}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'SqlServerWriter'

       Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}

       Writer Instance Id: {66ee8e0d-4a47-4fdd-9125-e16009c40998}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'MSDEWriter'

       Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}

       Writer Instance Id: {fe851803-0d54-4d4c-8462-1e2cd5f3efec}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'Registry Writer'

       Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}

       Writer Instance Id: {d76a01e2-086d-444e-8b92-d2b47a8b3f3a}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'Event Log Writer'

       Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}

       Writer Instance Id: {924394d0-0d24-47a4-9f60-6225ad1a8bd1}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'COM+ REGDB Writer'

       Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}

       Writer Instance Id: {0126ddb7-8fff-40c8-9107-2a4d85be4e6f}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'IIS Metabase Writer'

       Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}

       Writer Instance Id: {a953682e-f6eb-42f6-b1f7-d24ca1fb7262}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'WMI Writer'

       Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}

       Writer Instance Id: {807ddbe0-679a-4e6d-b5aa-ccede6ba94d2}

       State: [1] Stable

       Last error: No error

     

    Writer name: 'BITS Writer'

       Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}

       Writer Instance Id: {cac67565-ecfd-40c3-8987-a69d2bd1fedb}

       State: [1] Stable

       Last error: No error

    C:\>vssadmin list shadows

    vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool

    (C) Copyright 2001 Microsoft Corp.

     

    No items found that satisfy the query.


    3. File server protection, however each of servers are there is SQL. But we not backuping the SQL at this moment, yet.

    4. This file server hosted on Windows Server 2003 R2, I don't think SIS is enabled. I guessing SIS only avaliable on Windows Storage Server 2003 R2 (WSS 2003 R2) & Windows Unified Data Storage Server 2003 (WUDSS 2003), right?

    5. We set TEAMING with 2 LAN joined together (1Gbps x 2) on target machine and DPM 2010 server. On "Windows task manager > Networking" seem only use up below 12.5% (as now).




    • Edited by Oneoa Wednesday, February 15, 2012 2:59 AM
    Wednesday, February 15, 2012 2:53 AM
  • Hello, 
     

    The DPM server uses the protected server VSS to take the snapshot before transferring the data to DPM. If this piece breaks then DPM is just a victim.


    I believe that the 32-bit VSS limitation is the problem. This is due mainly in regards to the volume shadow copy service non-paged pool limitation for 32bit.  In regards to protected servers, memory problems can occur on the protected systems when there is not enough non-paged pool memory available to kernel components.  32bit systems have a maximum non-paged pool size that is 256MB regardless of how much RAM you have installed. The memory space is shared by other components such as volsnap, antivirus etc....

    The limitation is as this:

    32bit
    Paged Pool        470 MB
    NonPaged Pool  265MB

    64bit
    Paged Pool       128 gigs
    NonPaged Pool 128 gigs


    After a while the memory can become fragmented and eventually give insufficient resources errors. Such as the volsnap errors you are seeing above. There are some vss and volsnap updates that may be of use to you. I suggest you open up a support ticket with Microsoft for our performance team to assess those errors via perfmon counters usually. They would be the better qaulified to tell you if we have any VSS or Volsnap updates that may apply to your system or not. 

    In some case if you don't have enough free space for the page file to grow you can also recieve some of those errors as well. This would be the first thing I'd check. 


    Additional Resources
    http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx
    http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx

    Thanks,
    Shane

                                                                              
    • Edited by ShaneB. _ Wednesday, February 15, 2012 4:31 PM
    • Proposed as answer by ShaneB. _ Thursday, February 16, 2012 12:50 AM
    Wednesday, February 15, 2012 4:31 PM
  • Question is if we going to upgrade Windows Server 2003 R2 32-Bit to Windows Server 2008 R2 64-Bit. Would that solve the problem? What the impact upgrading rather then migration over to new server?
    Thursday, February 16, 2012 1:47 AM
  • Hello,

    64-bit machines most definitly have a larger paged pool and non-paged pool space which does for the most part alleviate the "insufficient non-paged memory pool" errors. Again, if VSS and Volsnap are having issues then DPM becomes the victim and we rely on those components. 

    Since this is 2003 you can re-registers the VSS dlls and there are a few VSS and Volsnap hotfixes that may help out some, but I'd let our performance team make that call.

    You can not upgrade from 32bit to 64bit.

    http://windows.microsoft.com/en-US/windows-vista/32-bit-and-64-bit-Windows-frequently-asked-questions 
    "If you are currently running a 32-bit version of Windows, you can only perform an upgrade to another 32-bit version of Windows. Similarly, if you are running a 64-bit version of Windows, you can only perform an upgrade to another 64-bit version of Windows."

    http://support.microsoft.com/kb/951041
    "Upgrades can be performed only between versions  of Windows that share the same architecture, such as x86 to x86, x64 to x64, or IA-64 to IA-64.  For example, you cannot upgrade x86 versions of Windows to x64 versions. "

    If you move to the 64bit platform, then I'd suggest you update it with the latest VSS and Volsnap patches availible as well. 

    Personally, going forward I would:
    a.) Check the free space you have on the drive.
    b.) Make sure you have a good size page file.  At least 1.5xRAM installed.
    c.) Remove any 3rd party filter drivers not needed.
    d.)Perform a windows update scan on this server and update anything related to VSS or volsnap.

    Once all of this is done, test your backup again and see if the same VSS or Volsnap errors are thrown.  If they are open up a support incident with our Performance team and let them assess the server and let them conclude if VSS needs to be re-registered, or if you have the latest VSS\Volsnap or if this server hitting resource thresholds. 

    Thanks
    Shane

    • Proposed as answer by ShaneB. _ Thursday, March 01, 2012 12:48 PM
    Thursday, February 16, 2012 1:30 PM
  • Hello,

    Do you still have this issue?

    Regards, Shane Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, March 01, 2012 12:49 PM
  • Yes I still have this issue, technically best suggesting here is to use latest windows server version with 64bit. Mean to say none solving the problem.
    • Edited by Oneoa Friday, March 02, 2012 2:47 AM
    Friday, March 02, 2012 2:39 AM
  • Hello,

    The paged pool and non-paged pool limitation is a x86 architecture limitation. Oneoa, I'd suggest to open up a support incident with our Performance team to look at the other VSS and volsnap issues. 

     

    Regards, Shane Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. This posting is provided "AS IS" with no warranties, and confers no rights.

    Friday, March 02, 2012 4:46 PM