locked
Errors after recovering SharePoint 2007 from DB crash RRS feed

  • Question

  • Hello

    First of all I want to apologize for the possible grammatical errors in text - I'm not a native English speaker.

    Now for the question.

    Short description is - after recovering SharePoint 2007 databases (MSSQL 2005) from backups on another server after original server crash the system event log is flooded with error that says "Could not find stored procedure 'dbo.proc_MSS_Cleanup'" and originates from OWSTIMER.EXE process. The errors look like this

    Process - OWSTIMER.EXE (0x0944)
    TID  - 0x09C4
    Area  - Windows SharePoint Services
    Category - Database
    EventID - 880i
    Level - High
    Message - System.Data.SqlClient.SqlException:
    Could not find stored procedure 'dbo.proc_MSS_Cleanup'.
    at System.Data.SqlClient.SqlConnection.OnError(SqlException exception,
    Boolean breakConnection)
    at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(
    TdsParserStateObject stateObj)
    at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior,
    SqlCommand cmdHandler, SqlDataReader dataStream,
    BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
    at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds,
    RunBehavior runBehavior, String resetOptionsString)
    at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(
    CommandBehavior cmdBehavior,
    RunBehavior runBehavior, Boolean returnStream, Boolean async)
    at System.Data.SqlClient.SqlCommand.Run...
    

    A quick look through search engines found posts about similar error, but they are either dated back to SharePoint 2007 beta-versions and deal with such an error occurring during single-server installation (like this one and this one - tried giving permissions, doesn't seem to work) or don't have a clean description of the reason of the error and ways of getting rid of it.

    Long description with chronology follows:

    Last week our (the company I work for) customer had a massive disaster in their IT-infrastructure - their SAN disk array broke and brought down with itself several systems, including the SharePoint 2007 portal we developed and deployed - MSSQL cluster died because both Windows cluster quorum disk and shared disk with MSSQL data files were located on that array. We can not restore cluster right now because that SAN still is not working correctly and the second SAN array this customer has is running almost at the full capacity.

    Since the customer has rather intensive document flow and wanted the portal back as soon as possible they provided us with a server we could use and so this weekend we installed MSSQL 2005 on that server (single installation, not a cluster, and data files located on the internal HDD - a temporary move until they fix their array).

    After installing MSSQL we restored portal's databases from backups. The restoration process was as following - manually created DBs with needed names, then restored backups into them, then reconfigured routing and DNS so virtual IP and domain name used by cluster would point to the new server (yes, I guess that it is not a proper way, but we were short on time).

    Now the portal seems to work again, but there is one problem - system event log is flooded with error that says that stored procedure dbo.proc_MSS_Cleanup can not be found.

    Also after recovery there was a problem with importing new users from AD -  it just seemed to either hang or run infinitely, but form some reason it resolved by itself.

    If it's any help, original farm configuration was - 2x Web-servers with load balancing, 2x DB servers in failover cluster with shared disk on SAN array. All servers running under Windows 2003 x64.

    Now Web-servers still work in load-balanced pair and communicate with temporary DB server (also Win2003 x64, MSSQL2005).

    I suppose we could have done something wrong during either MSSQL installation or backup restoration process.

    Thanks in advance

    Wednesday, January 19, 2011 10:11 PM

Answers

  • Wow - I'm not jealous of your situation! :-)  I'm curious if you turned diagnostic logging on for your SharePoint farm.  If you did, does the ULS log give you any information about what specific database is missing this stored procedure?  If you can figure out which database(s) is missing the stored procedure, you might be able to detach the database from SharePoint (Central Administration --> Application Management | Manage Databases --> Remove database) then re-attach it to SharePoint.  When you do this procedure, upon re-attach, SharePoint checks the database to make sure everything is there that SharePoint needs.  If anything is missing, it will update the database at that time.
    Ann
    • Marked as answer by Leoyi Sun Wednesday, February 9, 2011 2:48 AM
    Wednesday, February 2, 2011 12:51 AM

All replies

  • Wow - I'm not jealous of your situation! :-)  I'm curious if you turned diagnostic logging on for your SharePoint farm.  If you did, does the ULS log give you any information about what specific database is missing this stored procedure?  If you can figure out which database(s) is missing the stored procedure, you might be able to detach the database from SharePoint (Central Administration --> Application Management | Manage Databases --> Remove database) then re-attach it to SharePoint.  When you do this procedure, upon re-attach, SharePoint checks the database to make sure everything is there that SharePoint needs.  If anything is missing, it will update the database at that time.
    Ann
    • Marked as answer by Leoyi Sun Wednesday, February 9, 2011 2:48 AM
    Wednesday, February 2, 2011 12:51 AM
  • Thanks a lot for your suggestions, abui!

    We carefully examined logs and found out that it was one of the search databases that OWSTIMER complained about. Examining its contents showed that it indeed didn't have such stored procedure. Looking into another search-related databases showed that all of them have this procedure.

    Then we looked into our SharePoint test server and it appeared that such DBs should have dbo.proc_MSS_Cleanup procedure inside it. A couple of days ago the disk array was fixed so we restored MSSQL cluster and transferred data from the temporary server. Afterwards we have examined databases and the procedure was there.

    So it seems that we had messed when setting up temporary server and restoring backups into it. Although I don't understand how it was possible to restore backup without some parts of it. Oh well, I guess I'll have to research on the details of how MSSQL performs backup and restore.

    Once again, thank you.

    P.S.

    Sorry for not replying sooner, we were rather busy this week.

    Friday, February 11, 2011 9:24 AM