locked
VSS errors for SQL Server cause failing backups RRS feed

  • Question

  • I've got an SBS 2008 server that has a recurring issue with backups related the the SQL Server VSS writer.

    The errors seem to start with Event 9001

    The log for database 'xm8_27' is not available. Check the event log for related error messages. Resolve any errors and restart the database.

    Followed up with 3041

    BACKUP failed to complete the command BACKUP DATABASE xm8_27. Check the backup application log for detailed messages.

    Event 1 from SQL VDI

    SQLVDI: Loc=TriggerAbort. Desc=invoked. ErrorCode=(0). Process=3684. Thread=13492. Server. Instance=XACTWARE. VD=Global\{B9E65BDC-0508-4D73-BF4E-E639004C000D}9_SQLVDIMemoryName_0.

    Event 8193:

    SQL writer error: Unexpected error calling routine IClientVirtualDevice::GetCommand.  hr = 0x80770004.

    I reboot, all is fine for a few days, backups work fine and then Boom, I start getting these errors again. I can't keep rebooting every few days, I need backups to be reliable. VSS in general seems to be fatally flawed, with a single minor error killing the entire system until a reboot, and not allowing ANY part of a backup to proceed due to one minor subsystem failure.

    If I stop the SQL VSS Writer, I can backup but get a warning about it.

    So I have a few questions:

    How can I resolve this SQL backup error once and for all without rebooting the server?

    How can I force backups to continue even though a single VSS sub component has failed?

    How can I restart the VSS components without resorting to rebooting the entire server when they are stuck in Failed or Waiting for Completion? There has got to be a way. Rebooting the server can not be the answer to everything.

    Here is the output of vssadmin list writers:

    vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
    (C) Copyright 2001-2005 Microsoft Corp.

    Writer name: 'ASR Writer'
       Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
       Writer Instance Id: {9b661b56-738d-4853-8512-ab60bfe6d40e}
       State: [1] Stable
       Last error: No error

    Writer name: 'SqlServerWriter'
       Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
       Writer Instance Id: {88029c64-f3aa-4e57-8542-77da948300f4}
       State: [8] Failed
       Last error: Non-retryable error

    Writer name: 'System Writer'
       Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
       Writer Instance Id: {b091ada8-f28b-49b1-b3e3-572227ddea8c}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'SharePoint Services Writer'
       Writer Id: {c2f52614-5e53-4858-a589-38eeb25c6184}
       Writer Instance Id: {d863ec1e-fbc5-4b2a-8465-556e421c6c82}
       State: [1] Stable
       Last error: No error

    Writer name: 'FSRM Writer'
       Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}
       Writer Instance Id: {51005bbd-02f3-4aea-9aab-ee18b9c06a44}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'NPS VSS Writer'
       Writer Id: {35e81631-13e1-48db-97fc-d5bc721bb18a}
       Writer Instance Id: {739f2abd-c9ac-4343-a653-581753c7ffbd}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'Microsoft Exchange Writer'
       Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
       Writer Instance Id: {c37f45ae-9710-4171-91f6-eea6468172da}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'SPSearch VSS Writer'
       Writer Id: {57af97e4-4a76-4ace-a756-d11e8f0294c7}
       Writer Instance Id: {21eda2e6-c571-4a7f-8c1a-8b50bb3d2c0c}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'WMI Writer'
       Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
       Writer Instance Id: {7bf2f68d-a6ec-47c4-a63f-3de29dc25a61}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'BITS Writer'
       Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
       Writer Instance Id: {9c90973f-517c-44b8-ba69-e608a191bbe9}
       State: [1] Stable
       Last error: No error

    Writer name: 'Shadow Copy Optimization Writer'
       Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
       Writer Instance Id: {c3aeffb3-c2e6-4670-b853-b041c368d745}
       State: [1] Stable
       Last error: No error

    Writer name: 'TS Gateway Writer'
       Writer Id: {368753ec-572e-4fc7-b4b9-ccd9bdc624cb}
       Writer Instance Id: {52efe6fe-6286-481b-8af6-8ad470e97f55}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'FRS Writer'
       Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
       Writer Instance Id: {476a7ae6-8864-4e1d-ad25-0c4b48f3f70b}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'Registry Writer'
       Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
       Writer Instance Id: {714d824b-87b6-4ed8-ad13-c20a8a101e08}
       State: [1] Stable
       Last error: No error

    Writer name: 'NTDS'
       Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
       Writer Instance Id: {0cf69da9-cc30-4211-b479-105b54d7f554}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'COM+ REGDB Writer'
       Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
       Writer Instance Id: {97dbb7fb-49af-4257-8fd5-0fbeb810e86f}
       State: [1] Stable
       Last error: No error

    Writer name: 'IIS Config Writer'
       Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
       Writer Instance Id: {d2762ea3-1814-4658-8138-d5f9b7d1ff71}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'Dhcp Jet Writer'
       Writer Id: {be9ac81e-3619-421f-920f-4c6fea9e93ad}
       Writer Instance Id: {53d8ce60-9a60-42d6-aa29-48f8d534fc9b}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'Certificate Authority'
       Writer Id: {6f5b15b5-da24-4d88-b737-63063e3a1f86}
       Writer Instance Id: {86fedb23-3064-4a9e-9c38-be44dfb448b6}
       State: [5] Waiting for completion
       Last error: No error

    Writer name: 'IIS Metabase Writer'
       Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
       Writer Instance Id: {9a4d6a02-8a2f-4e21-8498-f469d87eba7b}
       State: [5] Waiting for completion
       Last error: No error

     

    Thursday, May 9, 2013 1:06 PM

Answers

  • In my followup thread in the SQL forums, Event 9001 - log is not available during backups, it turns out that the database AutoClose property causes this problem. AutoClose should be set to False for all producion databases.

    I consider it a bug that VSS fails due to a commonly set and valid database option. Thanks!

    • Marked as answer by Walt Reed Thursday, May 30, 2013 11:34 AM
    Thursday, May 30, 2013 11:34 AM

All replies

  • Adding some additional information,

    Regarding the specific database listed, I was able to offline / online the DB which "fixed" the log referenced, and could then perform a manual BACKUP DATABASE successfully, but when I ran my full script that backs up all my databases, a different database had that exact same log error - a database that previously worked successfully a few minutes before. Offlining and onlining that one and I could run the database backup script successfully on all databases to completion, but the VSS writers are still Failed and I can't do a full server backup. This whole issue seems to hinge on something going wonky with the log file - but I can't see exactly what.

    Thursday, May 9, 2013 1:44 PM
  • Hi ,

    Thank you for posting your issue in the forum.

    I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

    Thank you for your understanding and support.

    Best Regards,

    Andy Qi

    TechNet Subscriber Support

    If you are TechNet Subscription user and have any feedback on our support quality, please send your feedback here.


    Andy Qi
    TechNet Community Support

    Monday, May 13, 2013 9:16 AM
  • Hi Walt,

    First of all, let’s talk about some information about VSS, based on my knowledge, VSS is a set of Component Object Model (COM) application programming interfaces (APIs) that provides standardized interfaces, enabling third-party backup and restoration software to centrally manage the backup and restore operations on a variety of applications. VSS also implements a framework that enables volume backups to be performed while applications on a system continue to write to the volumes. VSS has three components:

    Requestor—The application that requests the creation of a shadow copy.
    Provider—The interface that provides the functionality to actually make the shadow copy.
    Writer—Application-specific software that acts to ensure that application data is ready for shadow copy creation.

    Most of the backup software including third party backup software are calling VSS components for the backup, if there are some problem about the VSS components such as VSS writers, VSS providers, then it will bring lots of unexpected issues.


    Currently I suggest you install some hotfixes on the problematic server to upgrade the Volsnap.sys file version based on your OS edition.

    Besides, please restart some service via below steps:

    a. Type Services.msc in the Run Search box to open the Services MMC.
    b.  Right-click the following services one at a time. For each service, click Restart:

    COM+ Event System
    COM+ System Application
    Microsoft Software Shadow Copy Provider
    Volume Shadow Copy

    Note: You can also run the following command to restart the services.

    Net stop EventSystem    
    Net start EventSystem

    Net stop COMSysApp       
    Net start COMSysApp

    Net stop swprv  
    Net start swprv

    Net stop vss       
    Net start vss


    From above information you provided, we can see that there are lots of VSS writers had some problem, so we should resolve this issue first. 


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    • Marked as answer by Andy Qi Tuesday, May 21, 2013 8:39 AM
    • Unmarked as answer by Walt Reed Friday, May 24, 2013 12:45 PM
    Tuesday, May 14, 2013 9:09 AM
  • How can I resolve this SQL backup error once and for all without rebooting the server? ---- You need to understand that the culprit database is 'xm8_27' , this is not default SQL database instance that comes along with SBS. Which application is this database from , have you investigated from the application side?

    How can I restart the VSS components without resorting to rebooting the entire server when they are stuck in Failed or Waiting for Completion? There has got to be a way. Rebooting the server can not be the answer to everything. --- Peterson already explained that.

    Tuesday, May 14, 2013 1:12 PM
  • Yes, I perfectly understand that the SQL databases are being problematic. As you can see from my followup post to my original message, the "problem" database changes. I can detach and attach the database, and then the backup work some times, and fails on a DIFFERENT database other times. Lather, rinse, repeat. This is not an application issue, this is more of a SQL Server issue, but it's also a VSS issue, and a Windows Backup issue.

    How things SHOULD work with Windows Backup, is that when a particular VSS writer is problematic (such as the SQL VSS Writer for "A" particular database) it should log the error but - Continue! Yes indeed, the entire backup of all the other databases, filesystem, Exchange should not be aborted due to a minor issue with a single subsystem. Some is better than none, or rather, 99.98% of a backup is better than 0%. In my opinion (having been in IT before there was a product called Windows,) this is a Severity 1 bug to fail the entire operation on a minor fault.

    Tuesday, May 14, 2013 9:09 PM
  • Stopping the services listed and starting did not resolve the ability to backup if the SQL VSS writer is still in error.

    Do you have any recommendations on the specific hotfixes for Volsnap.sys for SBS 2008 SP2? (it's current in regards to all other updates)

    The version of volsnap.sys I have is dated 8/21/2012 - there is no version number on that particular file (as shown by properties / details tab)

    Tuesday, May 14, 2013 9:23 PM
  • Hi,

    Based on my research, 2748349 is the latest hotfix to upgrade the volsnap.sys to 6.0.6002.22913 version(published on 8/21/2012), so the version on the problematic server is the latest version.

    Now, please type the following commands from an elevated command prompt:

    Takeown /f %windir%\winsxs\temp\PendingRenames /a
    icacls %windir%\winsxs\temp\PendingRenames /grant "NT AUTHORITY\SYSTEM:(RX)"
    icacls %windir%\winsxs\temp\PendingRenames /grant "NT Service\trustedinstaller:(F)"
    icacls %windir%\winsxs\temp\PendingRenames /grant BUILTIN\Users:(RX)
    Takeown /f %windir%\winsxs\filemaps\* /a
    icacls %windir%\winsxs\filemaps\*.* /grant "NT AUTHORITY\SYSTEM:(RX)"
    icacls %windir%\winsxs\filemaps\*.* /grant "NT Service\trustedinstaller:(F)"
    icacls %windir%\winsxs\filemaps\*.* /grant BUILTIN\Users:(RX)

    net stop cryptsvc
    net start cryptsvc

    Type the following command to verify that the writers are now listed normally or not.
    vssadmin list writers

    Regards,


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    • Marked as answer by Andy Qi Tuesday, May 21, 2013 8:40 AM
    • Unmarked as answer by Walt Reed Friday, May 24, 2013 12:38 PM
    Wednesday, May 15, 2013 7:19 AM
  • The writers listed as normal, but the next backup failed again with the same errors listed in the original posts with the SQL transaction logs becoming inaccessible. I'm restarting this under the SQL forums as this seems to be primarily an issue caused by a SQL Server defect.
    Friday, May 24, 2013 12:38 PM
  • Hi Walt,

    Thanks for your update.

    Yes, if the error is related SQL, please submit another forum post on the SQL forum.

    Regards,


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    Saturday, May 25, 2013 7:15 AM
  • In my followup thread in the SQL forums, Event 9001 - log is not available during backups, it turns out that the database AutoClose property causes this problem. AutoClose should be set to False for all producion databases.

    I consider it a bug that VSS fails due to a commonly set and valid database option. Thanks!

    • Marked as answer by Walt Reed Thursday, May 30, 2013 11:34 AM
    Thursday, May 30, 2013 11:34 AM