Monday, July 26, 2010 9:12 AM
We appear to be experiencing problems with a corrupt mailstore, we are getting messages like the ones posted below below in the application event log.
We use outlook 2010 final and some rules are not working for any user, the rules that are working it is not showing new mails that go into folders.
At '26/07/2010 09:59:27' the Exchange store database 'Mailbox Database 1095862411' copy on this server detected corruption on the active copy of the database. To help identify the specific failure, consult the Event log on the server for other storage and "ExchangeStoreDb" events. Service recovery was attempted by failover to another copy, which was unsuccessful in restoring the service. Error: There is only one copy of this mailbox database (Mailbox Database 1095862411). Automatic recovery is not available.
Information Store (4772) Mailbox Database 1095862411: A bad page link (error -327) has been detected in a B-Tree (ObjectId: 1218, PgnoRoot: 27145) of database E:\Exchange 2010\Mailbox\Mailbox Database 1095862411\Mailbox Database 1095862411.edb (27354 => 274949, 274948).
Tuesday, July 27, 2010 4:23 PM
First off I would check your event logs to see how long this has been going on. This problem is frequently caused by a hardware failure or by improper configuration of the anti-virus product and indicates that the logical database structure has become corrupted. This may occur for one or more of the following reasons:
- Disk caching has not committed transactions to the hard disk and the server has stopped responding (crashed).
- Incorrect log files were replayed during a database restoration.
- The server has a defective hard disk controller. Note A defective hard disk controller is typically indicated by disk input/output (I/O) errors that are listed in the application and system logs.
- Database log files have been removed that were not fully committed to the database. (Typically caused by AV solution misconfiguration)
- Because logical database structure corruption is not detected by the backup program, you may not immediately become aware of the problem. It only appears when a user tries to access the page that the data is stored on which is what it sounds like is happening here.
- Check your event logs and try to figure out how long this has been going on.
- Check your backup system, have you been getting successful backups? When was the last known good backup? If its still showing as backing up properly see # 5 above, i.e. the backup process doesn't check the validity of the entire DB so it may show as 100% good even though the issue exists.
- Unless you can find some issue with your anti-virus product ore recently made some upgrade right before the issue started I would make an immediate plan to build a new server and migrate your users to it ASAP before the problem becomes worse.
- The only time I have determined not to move off the hardware is when everything was fine until a customer updated the AV solution, added more memory, new driver etc and they can then undo the upgrade that caused the problem. Even then however, AFTER correcting the issue I would create a new database and migrate the users into it because the existing database does have corruption in it.
- You can also look at using ESEUTIL /P to correct the errors, however BEWARE that this is a drastic action that could very well decimate the database and make it completely unusable and should only be undertaken AFTER MAKING A COPY, and ONLY if its your last ditch effort and ONLY after exporting the data from the EDB to PST so that you have an alternative recovery path.
- If you want a robust GUI driven way to export mailbox data from the Production Exchange server OR a way to repair the offline EDB and then extract the data to PST MSG or XML you should check out Lucid8's DigiScope http://www.lucid8.com/product/digiscope.asp
Wednesday, July 28, 2010 8:38 AM
TechNet Subscriber Support (http://technet.microsoft.com/en-us/subscriptions/ms788697.aspx)
If you have any feedback on our support, please contact email@example.com
Thursday, July 29, 2010 2:13 PM
It did appear to be down to a raid array which we have now resolved, this has been going on for about 2 weeks.
What is the best way to repair the DB?
Thursday, July 29, 2010 6:25 PM
You can take the database offline and run a ESEUTIL /P to repair the database, however, realize that depending upon the level of the damage in the EDB it could simply repair the few problem pages OR it could make the database unusable. From what you are telling me I would
- Make a backup of the EDB, LOG set
- Create a new Database
- Migrate your users over to the new database
- Basically get ALL of the users moved or all of them moved that you can do without error
- If you get errors on moving some of the mailboxes, then I would A: Export the users information to PST so that you have a backup and then B: try a /P against the database with the remaining users. Once the /P completes you will need to run an ESEUTIL /D and then remount the database
- Have the users check to see if they can access their mailboxes and if it looks like any data is missing
- Move the mailboxes to the new EDB and Kill the OLD one
- Keep the PST's around in case any of the problem users are missing data