none
Exchange 2010 SP1 Event id 9040 : Unable to save watermarks

    Question

  • Dear all,

    I'm getting quite periodically (each 2-5 minutes) this error in event log. Can anybody guide me what's wrong?


    Service MSExchangeMailSubmission. An exception has been handled on behalf of an assistant. A report will be sent for the following exception: Microsoft.Exchange.Assistants.AIGrayException ---> Microsoft.Exchange.Common.GrayException ---> Microsoft.Mapi.MapiExceptionPartialCompletion: MapiExceptionPartialCompletion: Unable to save watermarks. (hr=0x40680, ec=0)
    Diagnostic context:
        Lid: 1494    ---- Remote Context Beg ----
        Lid: 10804   StoreEc: 0xFFFFF9BF
        Lid: 25094 
        Lid: 13710   StoreEc: 0x89B    
        Lid: 14672   StoreEc: 0x40680  
        Lid: 2031    StoreEc: 0x40680  
        Lid: 18009   StoreEc: 0x40680  
        Lid: 1750    ---- Remote Context End ----
        Lid: 32361   StoreEc: 0x40680  
       at Microsoft.Mapi.MapiExceptionHelper.ThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, SafeExInterfaceHandle iUnknown, Exception innerException)
       at Microsoft.Mapi.MapiEventManager.SaveWatermarks(Watermark[] watermarks)
       at Microsoft.Exchange.Assistants.EventAccess.<>c__DisplayClassd.<SaveWatermarks>b__c()
       at Microsoft.Exchange.Assistants.EventAccess.CallEventManager(EventManagerFunction function)
       at Microsoft.Exchange.Assistants.EventControllerPrivate.UpdateIdleWatermarksForAssistant(Guid[] idleMailboxes, Guid assistantId)
       at Microsoft.Exchange.Assistants.EventControllerPrivate.DisposeOfIdleDispatchers()
       at Microsoft.Exchange.Assistants.EventController.<UpdateWatermarks>b__4()
       at Microsoft.Exchange.Assistants.Util.<>c__DisplayClass1.<CoreCatchMeIfYouCan>b__0()
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(TryDelegate tryDelegate, FilterDelegate filterDelegate, CatchDelegate catchDelegate)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Common.GrayException.ExceptionCatcher(Object exception)
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(TryDelegate tryDelegate, FilterDelegate filterDelegate, CatchDelegate catchDelegate)
       at Microsoft.Exchange.Common.GrayException.MapAndReportGrayExceptions(UserCodeDelegate tryCode)
       at Microsoft.Exchange.Assistants.Util.CoreCatchMeIfYouCan(CatchMe function)
       at Microsoft.Exchange.Assistants.Util.CatchMeIfYouCan(CatchMe function)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Assistants.Util.TraceAndThrow(CatchMe function, AIException aiException)
       at Microsoft.Exchange.Assistants.Util.CatchMeIfYouCan(CatchMe function)
       at Microsoft.Exchange.Assistants.Base.CatchMeIfYouCan(CatchMe function).
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="MSExchange Assistants" />
        <EventID Qualifiers="16388">9040</EventID>
        <Level>4</Level>
        <Task>1</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2010-12-01T14:24:55.000000000Z" />
        <EventRecordID>73855</EventRecordID>
        <Channel>Application</Channel>
        <Computer>euprg1nt501.das4net.com</Computer>
        <Security />
      </System>
      <EventData>
        <Data>MSExchangeMailSubmission</Data>
        <Data>Microsoft.Exchange.Assistants.AIGrayException ---&gt; Microsoft.Exchange.Common.GrayException ---&gt; Microsoft.Mapi.MapiExceptionPartialCompletion: MapiExceptionPartialCompletion: Unable to save watermarks. (hr=0x40680, ec=0)
    Diagnostic context:
        Lid: 1494    ---- Remote Context Beg ----
        Lid: 10804   StoreEc: 0xFFFFF9BF
        Lid: 25094 
        Lid: 13710   StoreEc: 0x89B    
        Lid: 14672   StoreEc: 0x40680  
        Lid: 2031    StoreEc: 0x40680  
        Lid: 18009   StoreEc: 0x40680  
        Lid: 1750    ---- Remote Context End ----
        Lid: 32361   StoreEc: 0x40680  
       at Microsoft.Mapi.MapiExceptionHelper.ThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, SafeExInterfaceHandle iUnknown, Exception innerException)
       at Microsoft.Mapi.MapiEventManager.SaveWatermarks(Watermark[] watermarks)
       at Microsoft.Exchange.Assistants.EventAccess.&lt;&gt;c__DisplayClassd.&lt;SaveWatermarks&gt;b__c()
       at Microsoft.Exchange.Assistants.EventAccess.CallEventManager(EventManagerFunction function)
       at Microsoft.Exchange.Assistants.EventControllerPrivate.UpdateIdleWatermarksForAssistant(Guid[] idleMailboxes, Guid assistantId)
       at Microsoft.Exchange.Assistants.EventControllerPrivate.DisposeOfIdleDispatchers()
       at Microsoft.Exchange.Assistants.EventController.&lt;UpdateWatermarks&gt;b__4()
       at Microsoft.Exchange.Assistants.Util.&lt;&gt;c__DisplayClass1.&lt;CoreCatchMeIfYouCan&gt;b__0()
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(TryDelegate tryDelegate, FilterDelegate filterDelegate, CatchDelegate catchDelegate)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Common.GrayException.ExceptionCatcher(Object exception)
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(TryDelegate tryDelegate, FilterDelegate filterDelegate, CatchDelegate catchDelegate)
       at Microsoft.Exchange.Common.GrayException.MapAndReportGrayExceptions(UserCodeDelegate tryCode)
       at Microsoft.Exchange.Assistants.Util.CoreCatchMeIfYouCan(CatchMe function)
       at Microsoft.Exchange.Assistants.Util.CatchMeIfYouCan(CatchMe function)
       --- End of inner exception stack trace ---
       at Microsoft.Exchange.Assistants.Util.TraceAndThrow(CatchMe function, AIException aiException)
       at Microsoft.Exchange.Assistants.Util.CatchMeIfYouCan(CatchMe function)
       at Microsoft.Exchange.Assistants.Base.CatchMeIfYouCan(CatchMe function)</Data>
      </EventData>
    </Event>

     


    Martin
    Wednesday, December 01, 2010 3:59 PM

Answers

  • Hi all,

    I tried to move all mailboxes to new database and delete old database and it helps. Strange watermarks error in event log stop appearing.

    But I found other strange thing, my database was pretty weird. When I count size of all mailboxes it was around 11 GB, but edb file was 60GB!!!   When I was migrating mailboxes, one mailbox was showing in moverequeststatistics  size of mailbox was 10GB, but after migration in new database mailbox size was only 600MB.

    When all mailboxes was migrated (even arbitration mailboxes), mailbox database cleaned (so no disconneted mailboxes showed in EMC), I tried to offline defragment edb file with eseutil, and after all that edb file was still 54GB without any mailbox at all!!!

    So there was really something strange in database which cause this errors. My solution was to create new databases and move mailboxes to new one.

     


    Martin
    • Marked as answer by matyke Friday, December 03, 2010 9:31 AM
    Friday, December 03, 2010 9:31 AM

All replies

  • I'm having the exact same error.  I've noted one other post with a person having this.  They moved the users to a different store, deleted and recreated it.  This person had things in a test lab and my machines are in production, so that's not really an option.

    I'd be interested to see what the answer is to this.

     


    ----Jer
    Wednesday, December 01, 2010 5:14 PM
  • From what I have been able to put together so far, the MAPI error 0x40680 means it was unable to delete something.  So this would definitely fall in line with the throttle based managed folders that I am currently working on.

    Now I just have to figure out why it can't delete the item.


    ----Jer
    Wednesday, December 01, 2010 7:42 PM
  • Hello Jeremy,

    I didn't create any managed folders policy, it's just fresh installation of exchange 2010 sp1 server. Is there possibility to identify which mailbox is affected with this issue?

     


    Martin
    Wednesday, December 01, 2010 8:09 PM
  • Martin;

    I'm not 100% sure this error is associated with the managed folders.  I didn't notice the error until I started configuring the managed folders.  It's very possible the error was there before.

    The hr=0x40680 error from what I can find through searching show an error deleting an item via MAPI.  That is how I connected the 2 together.  However, it would appear that other people are having problems with retention tag based managed folders not deleting messages as well.  So I could be completely wrong.

    I have been unable to determine which mailbox is causing the error so far.


    ----Jer
    Wednesday, December 01, 2010 8:44 PM
  • Hi all,

    I tried to move all mailboxes to new database and delete old database and it helps. Strange watermarks error in event log stop appearing.

    But I found other strange thing, my database was pretty weird. When I count size of all mailboxes it was around 11 GB, but edb file was 60GB!!!   When I was migrating mailboxes, one mailbox was showing in moverequeststatistics  size of mailbox was 10GB, but after migration in new database mailbox size was only 600MB.

    When all mailboxes was migrated (even arbitration mailboxes), mailbox database cleaned (so no disconneted mailboxes showed in EMC), I tried to offline defragment edb file with eseutil, and after all that edb file was still 54GB without any mailbox at all!!!

    So there was really something strange in database which cause this errors. My solution was to create new databases and move mailboxes to new one.

     


    Martin
    • Marked as answer by matyke Friday, December 03, 2010 9:31 AM
    Friday, December 03, 2010 9:31 AM