Answered by:
VSS errors for SQL Server cause failing backups
Question
-
I've got an SBS 2008 server that has a recurring issue with backups related the the SQL Server VSS writer.
The errors seem to start with Event 9001
The log for database 'xm8_27' is not available. Check the event log for related error messages. Resolve any errors and restart the database.
Followed up with 3041
BACKUP failed to complete the command BACKUP DATABASE xm8_27. Check the backup application log for detailed messages.
Event 1 from SQL VDISQLVDI: Loc=TriggerAbort. Desc=invoked. ErrorCode=(0). Process=3684. Thread=13492. Server. Instance=XACTWARE. VD=Global\{B9E65BDC-0508-4D73-BF4E-E639004C000D}9_SQLVDIMemoryName_0.
Event 8193:
SQL writer error: Unexpected error calling routine IClientVirtualDevice::GetCommand. hr = 0x80770004.
I reboot, all is fine for a few days, backups work fine and then Boom, I start getting these errors again. I can't keep rebooting every few days, I need backups to be reliable. VSS in general seems to be fatally flawed, with a single minor error killing the entire system until a reboot, and not allowing ANY part of a backup to proceed due to one minor subsystem failure.
If I stop the SQL VSS Writer, I can backup but get a warning about it.
So I have a few questions:
How can I resolve this SQL backup error once and for all without rebooting the server?
How can I force backups to continue even though a single VSS sub component has failed?
How can I restart the VSS components without resorting to rebooting the entire server when they are stuck in Failed or Waiting for Completion? There has got to be a way. Rebooting the server can not be the answer to everything.
Here is the output of vssadmin list writers:
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {9b661b56-738d-4853-8512-ab60bfe6d40e}
State: [1] Stable
Last error: No errorWriter name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {88029c64-f3aa-4e57-8542-77da948300f4}
State: [8] Failed
Last error: Non-retryable errorWriter name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {b091ada8-f28b-49b1-b3e3-572227ddea8c}
State: [5] Waiting for completion
Last error: No errorWriter name: 'SharePoint Services Writer'
Writer Id: {c2f52614-5e53-4858-a589-38eeb25c6184}
Writer Instance Id: {d863ec1e-fbc5-4b2a-8465-556e421c6c82}
State: [1] Stable
Last error: No errorWriter name: 'FSRM Writer'
Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}
Writer Instance Id: {51005bbd-02f3-4aea-9aab-ee18b9c06a44}
State: [5] Waiting for completion
Last error: No errorWriter name: 'NPS VSS Writer'
Writer Id: {35e81631-13e1-48db-97fc-d5bc721bb18a}
Writer Instance Id: {739f2abd-c9ac-4343-a653-581753c7ffbd}
State: [5] Waiting for completion
Last error: No errorWriter name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {c37f45ae-9710-4171-91f6-eea6468172da}
State: [5] Waiting for completion
Last error: No errorWriter name: 'SPSearch VSS Writer'
Writer Id: {57af97e4-4a76-4ace-a756-d11e8f0294c7}
Writer Instance Id: {21eda2e6-c571-4a7f-8c1a-8b50bb3d2c0c}
State: [5] Waiting for completion
Last error: No errorWriter name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {7bf2f68d-a6ec-47c4-a63f-3de29dc25a61}
State: [5] Waiting for completion
Last error: No errorWriter name: 'BITS Writer'
Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
Writer Instance Id: {9c90973f-517c-44b8-ba69-e608a191bbe9}
State: [1] Stable
Last error: No errorWriter name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {c3aeffb3-c2e6-4670-b853-b041c368d745}
State: [1] Stable
Last error: No errorWriter name: 'TS Gateway Writer'
Writer Id: {368753ec-572e-4fc7-b4b9-ccd9bdc624cb}
Writer Instance Id: {52efe6fe-6286-481b-8af6-8ad470e97f55}
State: [5] Waiting for completion
Last error: No errorWriter name: 'FRS Writer'
Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
Writer Instance Id: {476a7ae6-8864-4e1d-ad25-0c4b48f3f70b}
State: [5] Waiting for completion
Last error: No errorWriter name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {714d824b-87b6-4ed8-ad13-c20a8a101e08}
State: [1] Stable
Last error: No errorWriter name: 'NTDS'
Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
Writer Instance Id: {0cf69da9-cc30-4211-b479-105b54d7f554}
State: [5] Waiting for completion
Last error: No errorWriter name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {97dbb7fb-49af-4257-8fd5-0fbeb810e86f}
State: [1] Stable
Last error: No errorWriter name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {d2762ea3-1814-4658-8138-d5f9b7d1ff71}
State: [5] Waiting for completion
Last error: No errorWriter name: 'Dhcp Jet Writer'
Writer Id: {be9ac81e-3619-421f-920f-4c6fea9e93ad}
Writer Instance Id: {53d8ce60-9a60-42d6-aa29-48f8d534fc9b}
State: [5] Waiting for completion
Last error: No errorWriter name: 'Certificate Authority'
Writer Id: {6f5b15b5-da24-4d88-b737-63063e3a1f86}
Writer Instance Id: {86fedb23-3064-4a9e-9c38-be44dfb448b6}
State: [5] Waiting for completion
Last error: No errorWriter name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {9a4d6a02-8a2f-4e21-8498-f469d87eba7b}
State: [5] Waiting for completion
Last error: No errorThursday, May 9, 2013 1:06 PM
Answers
-
In my followup thread in the SQL forums, Event 9001 - log is not available during backups, it turns out that the database AutoClose property causes this problem. AutoClose should be set to False for all producion databases.
I consider it a bug that VSS fails due to a commonly set and valid database option. Thanks!
- Marked as answer by Walt Reed Thursday, May 30, 2013 11:34 AM
Thursday, May 30, 2013 11:34 AM
All replies
-
Adding some additional information,
Regarding the specific database listed, I was able to offline / online the DB which "fixed" the log referenced, and could then perform a manual BACKUP DATABASE successfully, but when I ran my full script that backs up all my databases, a different database had that exact same log error - a database that previously worked successfully a few minutes before. Offlining and onlining that one and I could run the database backup script successfully on all databases to completion, but the VSS writers are still Failed and I can't do a full server backup. This whole issue seems to hinge on something going wonky with the log file - but I can't see exactly what.
Thursday, May 9, 2013 1:44 PM -
Hi ,
Thank you for posting your issue in the forum.
I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.
Thank you for your understanding and support.
Best Regards,
Andy Qi
If you are TechNet Subscription user and have any feedback on our support quality, please send your feedback here.
Andy Qi
TechNet Community SupportMonday, May 13, 2013 9:16 AM -
Hi Walt,
First of all, let’s talk about some information about VSS, based on my knowledge, VSS is a set of Component Object Model (COM) application programming interfaces (APIs) that provides standardized interfaces, enabling third-party backup and restoration software to centrally manage the backup and restore operations on a variety of applications. VSS also implements a framework that enables volume backups to be performed while applications on a system continue to write to the volumes. VSS has three components:
Requestor—The application that requests the creation of a shadow copy.
Provider—The interface that provides the functionality to actually make the shadow copy.
Writer—Application-specific software that acts to ensure that application data is ready for shadow copy creation.Most of the backup software including third party backup software are calling VSS components for the backup, if there are some problem about the VSS components such as VSS writers, VSS providers, then it will bring lots of unexpected issues.
Currently I suggest you install some hotfixes on the problematic server to upgrade the Volsnap.sys file version based on your OS edition.Besides, please restart some service via below steps:
a. Type Services.msc in the Run Search box to open the Services MMC.
b. Right-click the following services one at a time. For each service, click Restart:COM+ Event System
COM+ System Application
Microsoft Software Shadow Copy Provider
Volume Shadow CopyNote: You can also run the following command to restart the services.
Net stop EventSystem
Net start EventSystemNet stop COMSysApp
Net start COMSysAppNet stop swprv
Net start swprvNet stop vss
Net start vss
From above information you provided, we can see that there are lots of VSS writers had some problem, so we should resolve this issue first.Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Tuesday, May 14, 2013 9:09 AM -
How can I resolve this SQL backup error once and for all without rebooting the server? ---- You need to understand that the culprit database is 'xm8_27' , this is not default SQL database instance that comes along with SBS. Which application is this database from , have you investigated from the application side?
How can I restart the VSS components without resorting to rebooting the entire server when they are stuck in Failed or Waiting for Completion? There has got to be a way. Rebooting the server can not be the answer to everything. --- Peterson already explained that.
Tuesday, May 14, 2013 1:12 PM -
Yes, I perfectly understand that the SQL databases are being problematic. As you can see from my followup post to my original message, the "problem" database changes. I can detach and attach the database, and then the backup work some times, and fails on a DIFFERENT database other times. Lather, rinse, repeat. This is not an application issue, this is more of a SQL Server issue, but it's also a VSS issue, and a Windows Backup issue.
How things SHOULD work with Windows Backup, is that when a particular VSS writer is problematic (such as the SQL VSS Writer for "A" particular database) it should log the error but - Continue! Yes indeed, the entire backup of all the other databases, filesystem, Exchange should not be aborted due to a minor issue with a single subsystem. Some is better than none, or rather, 99.98% of a backup is better than 0%. In my opinion (having been in IT before there was a product called Windows,) this is a Severity 1 bug to fail the entire operation on a minor fault.
Tuesday, May 14, 2013 9:09 PM -
Stopping the services listed and starting did not resolve the ability to backup if the SQL VSS writer is still in error.
Do you have any recommendations on the specific hotfixes for Volsnap.sys for SBS 2008 SP2? (it's current in regards to all other updates)
The version of volsnap.sys I have is dated 8/21/2012 - there is no version number on that particular file (as shown by properties / details tab)
Tuesday, May 14, 2013 9:23 PM -
Hi,
Based on my research, 2748349 is the latest hotfix to upgrade the volsnap.sys to 6.0.6002.22913 version(published on 8/21/2012), so the version on the problematic server is the latest version.
Now, please type the following commands from an elevated command prompt:
Takeown /f %windir%\winsxs\temp\PendingRenames /a
icacls %windir%\winsxs\temp\PendingRenames /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\temp\PendingRenames /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\temp\PendingRenames /grant BUILTIN\Users:(RX)
Takeown /f %windir%\winsxs\filemaps\* /a
icacls %windir%\winsxs\filemaps\*.* /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\filemaps\*.* /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\filemaps\*.* /grant BUILTIN\Users:(RX)net stop cryptsvc
net start cryptsvcType the following command to verify that the writers are now listed normally or not.
vssadmin list writersRegards,
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Wednesday, May 15, 2013 7:19 AM -
The writers listed as normal, but the next backup failed again with the same errors listed in the original posts with the SQL transaction logs becoming inaccessible. I'm restarting this under the SQL forums as this seems to be primarily an issue caused by a SQL Server defect.Friday, May 24, 2013 12:38 PM
-
Hi Walt,
Thanks for your update.
Yes, if the error is related SQL, please submit another forum post on the SQL forum.
Regards,
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Saturday, May 25, 2013 7:15 AM -
In my followup thread in the SQL forums, Event 9001 - log is not available during backups, it turns out that the database AutoClose property causes this problem. AutoClose should be set to False for all producion databases.
I consider it a bug that VSS fails due to a commonly set and valid database option. Thanks!
- Marked as answer by Walt Reed Thursday, May 30, 2013 11:34 AM
Thursday, May 30, 2013 11:34 AM
