none
ActiveSync Error and Timeout Event ID 1016

    Question

  • Since installing Cumulative Update 10 on my Exchange Server 2010 SP3 system, I have experienced two events that have impacted my users with ActiveSync.  Apparently, after the update, ActiveSync will try to open a single person's mailbox (called firstname lastname in the error data) and is unable to do so.  ActiveSync thinks the server is unavailable due to load (it is not) and turns off ActiveSync for 60 seconds.  When ActiveSync turns back on after 60 seconds,  all ActiveSync users receive all of their messages again on their devices.  The event occurs every 10 minutes until ActiveSync is turned off for the user firstname lastname.  Additionally, I turn off IMAP and POP3 for firstname lastname.  ActiveSync no longer reports th error and all is well.  After deleting the devices from firstname lastname's account, the devices resync with ActiveSync and no longer have the problem.

    I have never had this problem before the update.  I have two questions:

    1)  Can I change the timeout to a very short number?

    2)  Is this problem due to Cumulative Update 10?

    Below is the data from the event logs for Event ID 1016 (with personally identifiable info changed):

    Alert: Active Sync is temporarily suspending requests to an unresponsive mailbox server. Users with mailboxes on this server may not be able to sync until the server recovers.
    Source: ActiveSync - SSS-CAT-FRONTEND1 (Client Access) - DataCenter
    Path: SSS-CAT-FRONTEND1.domain.com;SSS-CAT-FRONTEND1 (Client Access) - DataCenter Last modified by: NT AUTHORITY\SYSTEM Last modified time: 8/17/2015 5:40:22 PM Alert description: Exchange ActiveSync has encountered repeated failures when it tries to access data on Mailbox server [SSS-CAT-DB01.domain.com]. It will temporarily stop making requests to the Mailbox server for [60] seconds to reduce load on that server. This delay may occur if the Mailbox server is overloaded. If this event is logged frequently, review the Application log on this server and the Mailbox server noted above for other events that could indicate the root cause of performance problems.
    Additional information:
    "serverFQDN=SSS-CAT-DB01.domain.com
    Error 0:

    ErrorTimeStamp:
    8/17/2015 5:34:18 PM
    Exception:
    --- Exception start ---
    Exception type: Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException
    Exception message: Cannot open mailbox /o=SS-Exchange/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=Firstname Lastname.
    Exception level: 0
    Exception stack trace:    at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Initialize(MapiStore linkedStore, LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, GenericIdentity auxiliaryIdentity)
       at Microsoft.Exchange.Data.Storage.MailboxSession.<>c__DisplayClass12.<CreateMailboxSession>b__10(MailboxSession mailboxSession)
       at Microsoft.Exchange.Data.Storage.MailboxSession.InternalCreateMailboxSession(LogonType logonType, ExchangePrincipal owner, CultureInfo cultureInfo, String clientInfoString, IAccountingObject budget, Action`1 initializeMailboxSession, InitializeMailboxSessionFailure initializeMailboxSessionFailure)
       at Microsoft.Exchange.Data.Storage.MailboxSession.CreateMailboxSession(LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, CultureInfo cultureInfo, String clientInfoString, PropertyDefinition[] mailboxProperties, IList`1 foldersToInit, GenericIdentity auxiliaryIdentity, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ConfigurableOpen(ExchangePrincipal mailbox, MailboxAccessInfo accessInfo, CultureInfo cultureInfo, String clientInfoString, LogonType logonType, PropertyDefinition[] mailboxProperties, InitializationFlags initFlags, IList`1 foldersToInit, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString, Boolean wantCachedConnection)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString)
       at Microsoft.Exchange.AirSync.Command.OpenMailboxSession(AirSyncUser user, Boolean shouldUseBudget)
       at Microsoft.Exchange.AirSync.Command.WorkerThread()
    Inner exception follows...
    Exception type: Microsoft.Mapi.MapiExceptionLogonFailed
    Exception message: MapiExceptionLogonFailed: Unable to open message store. (hr=0x80040111, ec=1010) Diagnostic context:
        Lid: 55847   EMSMDBPOOL.EcPoolSessionDoRpc called [length=640]
        Lid: 43559   EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=224][latency=0]
        Lid: 23226   --- ROP Parse Start ---
        Lid: 27962   ROP: ropLogon [254]
        Lid: 17082   ROP Error: 0x3F2     
        Lid: 26937  
        Lid: 21921   StoreEc: 0x3F2     
        Lid: 27962   ROP: ropExtendedError [250]
        Lid: 1494    ---- Remote Context Beg ----
        Lid: 26426   ROP: ropLogon [254]
        Lid: 56333  
        Lid: 6372    StoreEc: 0x80070005
        Lid: 24824  
        Lid: 4740    StoreEc: 0x80070005
        Lid: 30409   StoreEc: 0x80070005
        Lid: 19145   StoreEc: 0x3F2     
        Lid: 23241   StoreEc: 0x3F2     
        Lid: 32186  
        Lid: 8620    StoreEc: 0x3F2     
        Lid: 1750    ---- Remote Context End ----
        Lid: 26849  
        Lid: 21817   ROP Failure: 0x3F2     
        Lid: 26297  
        Lid: 16585   StoreEc: 0x3F2     
        Lid: 32441  
        Lid: 1706    StoreEc: 0x3F2     
        Lid: 24761  
        Lid: 20665   StoreEc: 0x3F2     
        Lid: 25785  
        Lid: 29881   StoreEc: 0x3F2     
    Exception level: 1
    Exception stack trace:    at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
       at Microsoft.Mapi.ExRpcConnection.OpenMsgStore(OpenStoreFlag storeFlags, String mailboxDn, Guid mailboxGuid, Guid mdbGuid, MapiStore msgStorePrivate, String& correctServerDn, ClientIdentityInfo clientIdentityAs, String userDnAs, String applicationId, CultureInfo cultureInfo)
       at Microsoft.Mapi.MapiStore.OpenMapiStore(String serverDn, String userDn, String mailboxDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, String httpProxyServerName, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, Boolean wantRedirect, String& correctServerDN, ClientIdentityInfo clientIdentity, String applicationId, Client xropClient, Boolean wantWebServices, Byte[] clientSessionInfo, TimeSpan connectionTimeout)
       at Microsoft.Mapi.MapiStore.OpenMailbox(String serverDn, String userDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, WindowsIdentity windowsIdentity, String applicationId)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
    --- Exception end ---
    Error 1:

    ErrorTimeStamp:
    8/17/2015 5:34:17 PM
    Exception:
    --- Exception start ---
    Exception type: Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException
    Exception message: Cannot open mailbox /o=SS-Exchange/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=firstname lastname.
    Exception level: 0
    Exception stack trace:    at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Initialize(MapiStore linkedStore, LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, GenericIdentity auxiliaryIdentity)
       at Microsoft.Exchange.Data.Storage.MailboxSession.<>c__DisplayClass12.<CreateMailboxSession>b__10(MailboxSession mailboxSession)
       at Microsoft.Exchange.Data.Storage.MailboxSession.InternalCreateMailboxSession(LogonType logonType, ExchangePrincipal owner, CultureInfo cultureInfo, String clientInfoString, IAccountingObject budget, Action`1 initializeMailboxSession, InitializeMailboxSessionFailure initializeMailboxSessionFailure)
       at Microsoft.Exchange.Data.Storage.MailboxSession.CreateMailboxSession(LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, CultureInfo cultureInfo, String clientInfoString, PropertyDefinition[] mailboxProperties, IList`1 foldersToInit, GenericIdentity auxiliaryIdentity, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ConfigurableOpen(ExchangePrincipal mailbox, MailboxAccessInfo accessInfo, CultureInfo cultureInfo, String clientInfoString, LogonType logonType, PropertyDefinition[] mailboxProperties, InitializationFlags initFlags, IList`1 foldersToInit, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString, Boolean wantCachedConnection)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString)
       at Microsoft.Exchange.AirSync.Command.OpenMailboxSession(AirSyncUser user, Boolean shouldUseBudget)
       at Microsoft.Exchange.AirSync.Command.WorkerThread()
    Inner exception follows...
    Exception type: Microsoft.Mapi.MapiExceptionLogonFailed
    Exception message: MapiExceptionLogonFailed: Unable to open message store. (hr=0x80040111, ec=1010) Diagnostic context:
        Lid: 55847   EMSMDBPOOL.EcPoolSessionDoRpc called [length=629]
        Lid: 43559   EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=224][latency=0]
        Lid: 23226   --- ROP Parse Start ---
        Lid: 27962   ROP: ropLogon [254]
        Lid: 17082   ROP Error: 0x3F2     
        Lid: 26937  
        Lid: 21921   StoreEc: 0x3F2     
        Lid: 27962   ROP: ropExtendedError [250]
        Lid: 1494    ---- Remote Context Beg ----
        Lid: 26426   ROP: ropLogon [254]
        Lid: 56333  
        Lid: 6372    StoreEc: 0x80070005
        Lid: 24824  
        Lid: 4740    StoreEc: 0x80070005
        Lid: 30409   StoreEc: 0x80070005
        Lid: 19145   StoreEc: 0x3F2     
        Lid: 23241   StoreEc: 0x3F2     
        Lid: 32186  
        Lid: 8620    StoreEc: 0x3F2     
        Lid: 1750    ---- Remote Context End ----
        Lid: 26849  
        Lid: 21817   ROP Failure: 0x3F2     
        Lid: 26297  
        Lid: 16585   StoreEc: 0x3F2     
        Lid: 32441  
        Lid: 1706    StoreEc: 0x3F2     
        Lid: 24761  
        Lid: 20665   StoreEc: 0x3F2     
        Lid: 25785  
        Lid: 29881   StoreEc: 0x3F2     
    Exception level: 1
    Exception stack trace:    at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
       at Microsoft.Mapi.ExRpcConnection.OpenMsgStore(OpenStoreFlag storeFlags, String mailboxDn, Guid mailboxGuid, Guid mdbGuid, MapiStore msgStorePrivate, String& correctServerDn, ClientIdentityInfo clientIdentityAs, String userDnAs, String applicationId, CultureInfo cultureInfo)
       at Microsoft.Mapi.MapiStore.OpenMapiStore(String serverDn, String userDn, String mailboxDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, String httpProxyServerName, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, Boolean wantRedirect, String& correctServerDN, ClientIdentityInfo clientIdentity, String applicationId, Client xropClient, Boolean wantWebServices, Byte[] clientSessionInfo, TimeSpan connectionTimeout)
       at Microsoft.Mapi.MapiStore.OpenMailbox(String serverDn, String userDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, WindowsIdentity windowsIdentity, String applicationId)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
    --- Exception end ---
    Error 2:

    ErrorTimeStamp:
    8/17/2015 5:34:17 PM
    Exception:
    --- Exception start ---
    Exception type: Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException
    Exception message: Cannot open mailbox /o=SS-Exchange/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=firstname lastname.
    Exception level: 0
    Exception stack trace:    at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Initialize(MapiStore linkedStore, LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, GenericIdentity auxiliaryIdentity)
       at Microsoft.Exchange.Data.Storage.MailboxSession.<>c__DisplayClass12.<CreateMailboxSession>b__10(MailboxSession mailboxSession)
       at Microsoft.Exchange.Data.Storage.MailboxSession.InternalCreateMailboxSession(LogonType logonType, ExchangePrincipal owner, CultureInfo cultureInfo, String clientInfoString, IAccountingObject budget, Action`1 initializeMailboxSession, InitializeMailboxSessionFailure initializeMailboxSessionFailure)
       at Microsoft.Exchange.Data.Storage.MailboxSession.CreateMailboxSession(LogonType logonType, ExchangePrincipal owner, DelegateLogonUser delegateUser, Object identity, OpenMailboxSessionFlags flags, CultureInfo cultureInfo, String clientInfoString, PropertyDefinition[] mailboxProperties, IList`1 foldersToInit, GenericIdentity auxiliaryIdentity, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ConfigurableOpen(ExchangePrincipal mailbox, MailboxAccessInfo accessInfo, CultureInfo cultureInfo, String clientInfoString, LogonType logonType, PropertyDefinition[] mailboxProperties, InitializationFlags initFlags, IList`1 foldersToInit, IAccountingObject budget)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString, Boolean wantCachedConnection)
       at Microsoft.Exchange.Data.Storage.MailboxSession.Open(ExchangePrincipal mailboxOwner, WindowsPrincipal authenticatedUser, CultureInfo cultureInfo, String clientInfoString)
       at Microsoft.Exchange.AirSync.Command.OpenMailboxSession(AirSyncUser user, Boolean shouldUseBudget)
       at Microsoft.Exchange.AirSync.Command.WorkerThread()
    Inner exception follows...
    Exception type: Microsoft.Mapi.MapiExceptionLogonFailed
    Exception message: MapiExceptionLogonFailed: Unable to open message store. (hr=0x80040111, ec=1010) Diagnostic context:
        Lid: 55847   EMSMDBPOOL.EcPoolSessionDoRpc called [length=629]
        Lid: 43559   EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=224][latency=0]
        Lid: 23226   --- ROP Parse Start ---
        Lid: 27962   ROP: ropLogon [254]
        Lid: 17082   ROP Error: 0x3F2     
        Lid: 26937  
        Lid: 21921   StoreEc: 0x3F2     
        Lid: 27962   ROP: ropExtendedError [250]
        Lid: 1494    ---- Remote Context Beg ----
        Lid: 26426   ROP: ropLogon [254]
        Lid: 56333  
        Lid: 6372    StoreEc: 0x80070005
        Lid: 24824  
        Lid: 4740    StoreEc: 0x80070005
        Lid: 30409   StoreEc: 0x80070005
        Lid: 19145   StoreEc: 0x3F2     
        Lid: 23241   StoreEc: 0x3F2     
        Lid: 32186  
        Lid: 8620    StoreEc: 0x3F2     
        Lid: 1750    ---- Remote Context End ----
        Lid: 26849  
        Lid: 21817   ROP Failure: 0x3F2     
        Lid: 26297  
        Lid: 16585   StoreEc: 0x3F2     
        Lid: 32441  
        Lid: 1706    StoreEc: 0x3F2     
        Lid: 24761  
        Lid: 20665   StoreEc: 0x3F2     
        Lid: 25785  
        Lid: 29881   StoreEc: 0x3F2     
    Exception level: 1
    Exception stack trace:    at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
       at Microsoft.Mapi.ExRpcConnection.OpenMsgStore(OpenStoreFlag storeFlags, String mailboxDn, Guid mailboxGuid, Guid mdbGuid, MapiStore msgStorePrivate, String& correctServerDn, ClientIdentityInfo clientIdentityAs, String userDnAs, String applicationId, CultureInfo cultureInfo)
       at Microsoft.Mapi.MapiStore.OpenMapiStore(String serverDn, String userDn, String mailboxDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, String httpProxyServerName, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, Boolean wantRedirect, String& correctServerDN, ClientIdentityInfo clientIdentity, String applicationId, Client xropClient, Boolean wantWebServices, Byte[] clientSessionInfo, TimeSpan connectionTimeout)
       at Microsoft.Mapi.MapiStore.OpenMailbox(String serverDn, String userDn, Guid guidMailbox, Guid guidMdb, String userName, String domainName, String password, ConnectFlag connectFlags, OpenStoreFlag storeFlags, CultureInfo cultureInfo, WindowsIdentity windowsIdentity, String applicationId)
       at Microsoft.Exchange.Data.Storage.MailboxSession.ForceOpen(MapiStore linkedStore)
    --- Exception end ---
    errorCount=3, backingOff=True".
    EventSourceName: MSExchange ActiveSync



    • Edited by LeslieQ Friday, August 21, 2015 6:24 PM
    Wednesday, August 19, 2015 2:31 PM