locked
Exchange 2010 SP2 IndexOutOfRangeException RRS feed

  • Question

  • I just upgraded our Exchange 2010 servers to SP2 a couple days ago and started seeing the following ASP.NET error pop up in the event log every few minutes on our CAS/HT Server; it's generating a ton of error reports. It doesn't seem to be affecting functionality at all, but it's quite annoying to see it filling up the log.
    Event code: 3005 
    Event message: An unhandled exception has occurred. 
    Event time: 12/9/2011 10:29:04 AM 
    Event time (UTC): 12/9/2011 3:29:04 PM 
    Event ID: 037085b1664e4c8286331d7250b57768 
    Event sequence: 2694951 
    Event occurrence: 1582 
    Event detail code: 0 
     
    Application information: 
        Application domain: /LM/W3SVC/1/ROOT/EWS-2-129677608322282356 
        Trust level: Full 
        Application Virtual Path: /EWS 
        Application Path: C:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\exchweb\EWS\ 
        Machine name: EXCH-CAS1 
     
    Process information: 
        Process ID: 6724 
        Process name: w3wp.exe 
        Account name: NT AUTHORITY\SYSTEM 
     
    Exception information: 
        Exception type: IndexOutOfRangeException 
        Exception message: Index was outside the bounds of the array. 

    I thought resetting the EWS virtual directory might fix it, but that didn't do anything. Anyone else having this problem?
    Friday, December 9, 2011 3:57 PM

Answers

  • Update (1/19/2012)
    An interim update (IU) is now available for this issue, please see the following KB article for more details on how to obtain that. Thanks for your patience while we worked to resolve this issue.

    Event ID 4999 is logged on an Exchange Server 2010 Client Access server (CAS)
    http://support.microsoft.com/kb/2665115

    --------------------------------------------------

    Update (1/12/2012)
    We've been able to identify the cause of this issue. At this time, it seems like if you have some special characters, like "(", ")" or ":" in the Exchange database name, then you can run into this issue. The workaround (rename the database to take out the special characters) mentioned below in some latest posts are not tested by Microsoft so we cannot recommend them to our customers. We are working diligently on a fix for this issue.

    --------------------------------------------------

    This is now a known issue with Exchange 2010 SP2. We have a bug open, which is being investigated for a fix. There is no ETA for a fix at this time. I'll post an update here, as soon as one is available. Customers who are experiencing this issue should call Microsoft Support (1-800-Microsoft) to open a support case (it'll be free of charge), so that when a fix is available, it can be provided to them. At this time, there is no known workaround for this issue. Thanks for your patience as we work to address this issue.

    Issue
    After you install Exchange 2010 SP2, 'MSExchangeServicesAppPool' may crash on startup. The w3wp process will keep running but may fail to service any EWS requests. This may result in the loss of OOF (Out of Office), Availability, Lync State features, and EWS clients such as Mac Outlook may not work properly.


    Sr. Program Manager, Product Quality, Exchange Client Access Server

    Tuesday, January 10, 2012 10:45 PM

All replies

  • I have same problem after Upgrade Exchange 2010 SP1 to Exchange 2010 SP2

     Log Name:      Application
    Source:        ASP.NET 2.0.50727.0
    Date:          12/12/2011 10:00:01 AM
    Event ID:      1309
    Task Category: Web Event
    Level:         Warning
    Keywords:      Classic
    User:          N/A
    Computer:      xxxxx
    Description:
    Event code: 3005
    Event message: An unhandled exception has occurred.
    Event time: 12/12/2011 10:00:01 AM
    Event time (UTC): 12/12/2011 3:00:01 AM
    Event ID:

    01f5efe65f2640e0847cea43060d0185
    Event sequence: 10597
    Event occurrence: 3144
    Event detail code: 0
     
    Application information:
        Application domain:

    /LM/W3SVC/1/ROOT/EWS-1-129680970032067457
        Trust level: Full
        Application Virtual Path: /EWS
        Application Path: C:\ExchangeServer\ClientAccess\exchweb\EWS\

    Machine name:  
    Process information:
        Process ID: 6812
        Process name: w3wp.exe
        Account name: NT AUTHORITY\SYSTEM
     
    Exception information:
        Exception

    type: IndexOutOfRangeException
        Exception message: Index was outside the bounds of the array.
     
    Request information:
        Request URL: xxxxx 
        Request path: /EWS/Exchange.asmx
        User host address: xxxx

        User: xxxxx   

    Is

    authenticated: True
        Authentication Type: Negotiate
        Thread account name: NT AUTHORITY\SYSTEM
     
    Thread information:
        Thread ID: 27
        Thread account name: NT

    AUTHORITY\SYSTEM
        Is impersonating: False
        Stack trace:    at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.SetBugdetSnapshotResourceInfo(String

    budgetResourcesString)
       at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.FetchThrottlingData()
       at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.Commit()
       at Microsoft.Exchange.Services.Global.Application_EndRequest(Object sender, EventArgs e)
       at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
       at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

    • Edited by Tamjy Monday, December 12, 2011 9:02 AM
    Monday, December 12, 2011 8:59 AM
  • We're having the same issues plus we're experiencing problems with Automatic Reply (former Out of Office) in Outlook 2007/2010. And it seems MacOS "Mail" clients are affected too, because no MacOS client can access mailbox since SP2.

    Any advice is of course appreciated. It looks like the same problem that Exchange 2007 was having and that was solved uninstalling .NET hotfixes, but that is not an option on Windows 2008 R2 server it seems.

    Monday, December 12, 2011 2:04 PM
  • Same errors here with SP2. Restart IIS and you can get the OOO box up once if you are fast. Or recycle the EWS worker pool.

    Tried installing a clean enw server, add SP2. It also fails. When can we expect a fix?

     

    Thread information:

    Thread ID: 20

    Thread account name: NT AUTHORITY\SYSTEM

    Is impersonating: False

    Stack trace: at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.SetBugdetSnapshotResourceInfo(String budgetResourcesString)

    at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.FetchThrottlingData()

    at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase.Commit()

    at Microsoft.Exchange.Services.Global.Application_EndRequest(Object sender, EventArgs e)

    at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

    at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

    Monday, December 12, 2011 8:29 PM
  • I've the same errors, mac's won't connect and the out of office is not working.

    I just tried to get the macs working by mapi, thats works, but the problem for the ooo still exists.

    Hope to see a fix soon.

    Thursday, December 15, 2011 10:44 AM
  • I've just received an information that there's about a "million" open bugs in SP2 and EWS and there isn't yet a fix for that (not even an internal one) but it's got a top priority so they're working hard on resolving this problem.

    I guess it's good to know that someone is working on it ;-)

    Thursday, December 15, 2011 2:03 PM
  • We have same problems on server upgraded to 2010 SP2 from SP1 RU6.
    Monday, December 19, 2011 1:05 PM
  • Outlook for mac 2011 seriously broken after upgrade to exchange 2010 SP2. (OSX 10.7.2, Outlook 14.1.4 111121)

    Similar EWS / IndexOutOfRangeException messages in server eventlog. Receiving email on mac is generally ok, Sending email from mac is hit and miss, getting "Mail could not be sent. Error code: -17890" with sometimes duplicate messages going out.

    This is very painfull. Look forward to fix soon.

    Monday, December 19, 2011 6:45 PM
  • Opened case with PSS.

    All client functions that are relying on EWS are affected;

    Outlook 2007/2010: Free Busy, OOF

    Oulook 2011 for mac : sending email (sometimes fails, sometimes sends duplicates), Free / busy, OOF

    only clients that are not affected are OWA and Activesync.

    Wednesday, December 21, 2011 12:22 AM
  • We have the same problem after installing Exchange 2010 SP2 on Exchange 2010 SP1 RU6.
    On the server installed role UM. On the client Lync 2010 error - "Failed to connect to Exchange Server" and do not work "Call Wia Work" on Mobility Client. (


    makuta
    Thursday, December 22, 2011 7:10 AM
  • Has anybody seen a solution for this issue - this is affecting my environment badly as we rely on EWS for Lync 2010 Integration for Missed Calls and Voicemail notifications

     

    Is there a workaround (Iknow you cannot uninstall a Service Pack) so is the only solution a new CAS install with SP1

     

    Wednesday, December 28, 2011 12:43 AM
  • Hi All,

    We are working on this issue, the fix will be included in the next updates of E2010 Sp2. Currently, there are no workarounds for this isssue.

     

     

    Thursday, December 29, 2011 6:33 AM
  • Hi Santhosh

     

    Does this mean for a fix we have to wait till the next rollup which im assuming is awile away yet

    As this is a major issue will there be any prerealease rollups released to fix thius beforehand??? I am not having any luck with the PSS call i have raised i have had no updates for over 1 week from the escalation team and as mentioned it is causing major issues with Lync and also users unable set OOF via Outlook

    Thursday, January 5, 2012 4:45 AM
  • Hi Maz,

    Yes, the SE team is working on this as a high prioerity bug and yet the build the fix. There is no ETA as of now.

    If you already have a PSS case open, you may recieve updates from the contact from PSS when the bug fix is completed. You then have an option to request for an Interim update without having to wait for the next public release of the update.

    Thank you

    Santhosh

     

    Thursday, January 5, 2012 5:40 AM
  • Well maybe the service pack should be taken down until a resolution has been found. I was unaware of this issue until I installed the service pack and it started.
    Thursday, January 5, 2012 5:49 AM
  • Hi,

    is the described behavior a general bug after the upgrade to SP2 ? In my test environment i can not reproduce even one of the errors. With Outlook 2007/2010 i can use oof as before and Outlook for MAC 2011 (MAC OS 10.7.2) can send and receive mails without a problem. Must there be any special circumstances to get the bugs ? We have for each server role a single virtual server.

    Thank you.

     

    Sven

    Thursday, January 5, 2012 7:51 AM
  • Hi Santhosh,

    I could really do with a workaround. Will uninstalling SP2 resolve the issue?

    Thursday, January 5, 2012 10:46 AM
  • I believe a notification is required to update the user community on this issue, i know various exchange engineers are waiting for the fix to be release and are fed up giving answers to their customers everyday :(

     

    Thanks

    Inderjit


    Inderjit
    Thursday, January 5, 2012 2:42 PM
  • I agree with all comments here if MS was aware of this bug there should have been some acknowledgement (looking through other forum posts people have noted this since 15 December i hate to think how many other users may be installing SP2 now and experiencing the same issue

    If there was some advisory if would have prevented a lot of issues (and calls to MS) especially as it seems to be a major issue with no easy fix or workaround in site

    How this was never picked up in testing by MS really surprises but with there recent record and releasing bug filled rollups for Exchange (SP1 Rollup4 anyone)  maybe it is not such a surprise - the days on me installing rollups and service packs upon release are long gone now after this issue

    P.S.

    I even recieved this snippit of an email from an MS escalation engineer, so i have no hope this will be fixed quickly

    The issue you’re facing is a known issue with EWS. Currently we are actively working with the Product Group to fix the issue, which is going to take some time.  I will let you know as soon as an update is available. 

    Friday, January 6, 2012 9:31 AM
  • Has anyone confirmed with MS that there's really isn't a workaround. Would uninstalling SP2 fix the issue? Unfortunately I haven't got an environment to test it and I'm worried that uninstalling the SP may make matters worse.

    Friday, January 6, 2012 11:44 AM
  • Cwica, Uninstall SP2 will completely uninstall Exchange and you will have to re-install... See http://technet.microsoft.com/en-us/library/bb629560.aspx 

    After you upgrade Exchange 2010 to a newer service pack, you can't uninstall the service pack to revert to the previous version. If you uninstall the service pack, you remove Exchange from the server. 

    Can hardly call that a workaround.

     

     

    Monday, January 9, 2012 9:10 PM
  • So we're in the hands of MS.  Does anyone have a timescale for the fix, I'm getting quite a bit of ear ache over this.
    Monday, January 9, 2012 9:24 PM
  • I have not received ETA for fix from MS. Last response on my PSS case was December 29;

    "We are consolidating a few other cases with the same symptom and have escalated to the proper group for traction. We are still testing the fix though. I’ve let the team know that you are willing to try out a private hotfix or an interim update, so hopefully we’ll have something for you soon!"

    Odds are that opening a case with MS will get you access to an interim fix sooner than fix beeing made available for public consumption.

    Monday, January 9, 2012 10:24 PM
  • Update (1/19/2012)
    An interim update (IU) is now available for this issue, please see the following KB article for more details on how to obtain that. Thanks for your patience while we worked to resolve this issue.

    Event ID 4999 is logged on an Exchange Server 2010 Client Access server (CAS)
    http://support.microsoft.com/kb/2665115

    --------------------------------------------------

    Update (1/12/2012)
    We've been able to identify the cause of this issue. At this time, it seems like if you have some special characters, like "(", ")" or ":" in the Exchange database name, then you can run into this issue. The workaround (rename the database to take out the special characters) mentioned below in some latest posts are not tested by Microsoft so we cannot recommend them to our customers. We are working diligently on a fix for this issue.

    --------------------------------------------------

    This is now a known issue with Exchange 2010 SP2. We have a bug open, which is being investigated for a fix. There is no ETA for a fix at this time. I'll post an update here, as soon as one is available. Customers who are experiencing this issue should call Microsoft Support (1-800-Microsoft) to open a support case (it'll be free of charge), so that when a fix is available, it can be provided to them. At this time, there is no known workaround for this issue. Thanks for your patience as we work to address this issue.

    Issue
    After you install Exchange 2010 SP2, 'MSExchangeServicesAppPool' may crash on startup. The w3wp process will keep running but may fail to service any EWS requests. This may result in the loss of OOF (Out of Office), Availability, Lync State features, and EWS clients such as Mac Outlook may not work properly.


    Sr. Program Manager, Product Quality, Exchange Client Access Server

    Tuesday, January 10, 2012 10:45 PM
  • The following workaround fixed the issue for me;

    1) Using Exchange Management Console; go into the properties of each mailbox database that contains special characters in it's name and change the name of the database to avoid round brackets (dash and space are ok, round brackets are especially bad)
    2) Using "Services" in Server manager on mailbox servers, restart the "Microsoft Exchange Throttling" service
    3) Using "IIS management" on Client Access servers, recycle the "MSExchangeServiecsAppPool"

    and to quote from Hitchhiker's Guide to the Galaxy: "Normality has been restored."

    • Proposed as answer by adenhaan Wednesday, January 11, 2012 10:53 PM
    Wednesday, January 11, 2012 10:29 PM
  • I have tested this fix on selected databases and this has fixed the issue straightaway my databases have as mentioned a bracket in the names removed them did all steps and all has come good again

    Well done MS weeks of waiting for a fix and a user has suggested a workaround that has seemed to fixed the issue

    Adenhann thanks so much for posting this

    I will report once i have applied this on a company wide basis and report back if any issues

     


    • Edited by maz124ss Wednesday, January 11, 2012 11:49 PM
    Wednesday, January 11, 2012 11:47 PM
  • So does this issue only apply to an exchange environment where the name of mailbox databases contain certain characters in them? like the round backets?
    Thursday, January 12, 2012 12:53 AM
  • Unreal...we were having problems with Mac Entourage clients performing various tasks in public folders.  The public folder database just so happens to have parenthesis in the name.  Followed your steps over an hour ago and I haven't seen an error since.

    Not sure how you figured this out, but many thanks for the solution!

    Thursday, January 12, 2012 1:25 AM
  • I sent MS PSS engineer an e-mail about the issue informing him of this fix and he came back with that just this morining that in fact MS have confirmed the issue has to do with special characters being in the database name below

    And according to the latest update I received today, this issue does have much to do with the database name. Characters like ),”, ‘:’, ‘(‘ are key delimiters to the budget resource string parsing logic in Exchange 2010

    He will be trying to get the prerelease hotfix soon but i am not waiting around for this and going through and changing my 60+ databases names tonight to fix issue now

    Also mentioned that public will get this fix in mid Feb with Rollup 1

    Very frustrating for such a small issue but oh well thankful at least the cause seems to have been identified anf there some fix in site now


    • Edited by maz124ss Thursday, January 12, 2012 1:51 AM
    Thursday, January 12, 2012 1:38 AM
  • Gents, I'm glad to hear that this relative straightforward workaround has helped you as well and gets you going again NOW instead of waiting another month for public hotfix/update rollup. Pls cut MS some slack... they have to follow various processes that take time (and you wouldn't want it any other way). Also to be honost I did not figgure this out all by myself, and praise goes to the PSS folks that I had the pleasure of working with. Feel free to mark my post as answer/helpfull. Andre.
    Thursday, January 12, 2012 3:27 AM
  • All I can say is WOW and Thank You.

    I was about to rebuild an entire exchange environment because of this issue and you saved me several days of work not to mention the political troubles this has caused over the past two weeks.

    Thursday, January 12, 2012 6:05 AM
  • Hi,

    we had the same problems:

    - OOF not working from Outlook (only in OWA)
    - Lync comm with Exchange not working properly (Comm errors and conv history not being written to Mailbox)
    - Mac users having problems with Emails being sent in Outlook 2011 several times because they got stucked in the outbox, problems with delayed syncing, etc.

    We got it resolved now thanks to adenhaan's Post. :-)

    We were using '(' and ')' in our database names - which turned out to be a problem only after this SP2 upgrade. Renaming the DBs by deleting these characters - can be done just online in DB's properties - and restarting the "MSExchangeServiecsAppPool" made everything work again.

    No idea who had the really bad idea to use these characters in the DB's name - ups, it was me... ;-)

    Thanks for the support!!!


    • Edited by ABBYYEU Friday, January 13, 2012 12:52 PM
    Thursday, January 12, 2012 11:47 AM
  • I am unable to access mailboxes with Mac Outlook clients after installing Exchange 2010 SP2. I only have two databases and neither of them have special characters so the solution above doesn't apply for me. I am getting this error below nonstop. This is very frustrating. Doesn't Microsoft test before releasing?

     

    Log Name:      Application
    Source:        System.ServiceModel 3.0.0.0
    Date:          1/16/2012 3:24:11 PM
    Event ID:      3
    Task Category: WebHost
    Level:         Error
    Keywords:      Classic
    User:          SYSTEM
    Computer:      mail.domain.com
    Description:
    WebHost failed to process a request.
     Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/32001227
     Exception: System.ServiceModel.ServiceActivationException: The service '/Autodiscover/autodiscover.xml' cannot be activated due to an exception during compilation.  The exception message is: This collection already contains an address with scheme http.  There can be at most one address per scheme in this collection.
    Parameter name: item. ---> System.ArgumentException: This collection already contains an address with scheme http.  There can be at most one address per scheme in this collection.
    Parameter name: item
       at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)
       at System.Collections.Generic.SynchronizedCollection`1.Add(T item)
       at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)
       at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)
       at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type serviceType, Uri[] baseAddresses)
       at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
       --- End of inner exception stack trace ---
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
     Process Name: w3wp
     Process ID: 2004

    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="System.ServiceModel 3.0.0.0" />
        <EventID Qualifiers="49154">3</EventID>
        <Level>2</Level>
        <Task>5</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2012-01-16T20:24:11.000000000Z" />
        <EventRecordID>185447</EventRecordID>
        <Channel>Application</Channel>
        <Computer>mail.bhme.com</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data>System.ServiceModel.ServiceHostingEnvironment+HostingManager/32001227</Data>
        <Data>System.ServiceModel.ServiceActivationException: The service '/Autodiscover/autodiscover.xml' cannot be activated due to an exception during compilation.  The exception message is: This collection already contains an address with scheme http.  There can be at most one address per scheme in this collection.
    Parameter name: item. ---&gt; System.ArgumentException: This collection already contains an address with scheme http.  There can be at most one address per scheme in this collection.
    Parameter name: item
       at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)
       at System.Collections.Generic.SynchronizedCollection`1.Add(T item)
       at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)
       at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)
       at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type serviceType, Uri[] baseAddresses)
       at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
       --- End of inner exception stack trace ---
       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
       at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)</Data>
        <Data>w3wp</Data>
        <Data>2004</Data>
      </EventData>
    </Event>

    Monday, January 16, 2012 8:46 PM
  • Hot Fix Interim Update has been released everyone, installed hotfix and issue all fixed

    Knowledge Base Article Below and issue description below

    http://support.microsoft.com/kb/2665115

    Friday, January 20, 2012 10:12 PM
  • I am having the same issues. I called Microsoft Support and they provided me with IU that was named "Exchange2010-KB2549255-x64-en". This did NOT correct the problems I am having. Did they provide me with the correct IU?
    Thursday, February 2, 2012 2:41 PM
  • I just received an email from Microsoft basically telling me they gave me the wrong IU. The correct IU to use is "Exchange2010-KB2665115-x64-en". I am planning on installing it tonight. They told me to install this on both my CAS/HUB server and my Mailbox server. Is it really neccessary to install on the Mailbox server?
    Thursday, February 2, 2012 3:51 PM
  • I applied the correct IU "Exchange2010-KB2665115-x64-en" to CAS/HUB and Mailbox servers and all problems were corrected.

    1. Why wasn't SP2 tested more thoroughly?

    2. Why couldn't Microsoft do a better job of making us aware of the issues.

    3. Why couldn't Microsoft Support provide the correct IU the first time?

    Some of my users were without email for 2 days.

    Friday, February 3, 2012 3:41 PM