none
EWS with O365 has been broken for the past 24 hours for us.

    Question

  • We have a relatively large application that imports emails from o365 via EWS, it uses a service account that checks all user's inboxes. at <g class="gr_ gr_27 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" data-gr-id="27" id="27">7:44am</g> on Wednesday morning we started seeing the error message for all users.

    The specified object was not found in the store., Can't connect to the mailbox of user Mailbox database <g class="gr_ gr_44 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" data-gr-id="44" id="44">guid</g>: [redacted guid] because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently.

    This process has been running smoothly for around a year.

    I have been told that there is a discrepancy between the GUID that o365 is returning via EWS and what it should be.

    The issue has been raised with Microsoft, who have fed back that due to GDPR reasons they're moving our data into Europe, and this is the likely cause, they were not sure how long this would take or say for certain whether this was the cause.

    I've been keeping a close eye on google but haven't found anybody else being affected by this.

    I'm wondering if anybody can answer two questions on this.

    1. Is anybody else experiencing this?

    2. Is there anything I can do? Our email system has been broken for 24 hours and I've run out of ideas for how to resolve quickly other than twiddling my thumbs. Any ideas would be appreciated. 24 hours seems like an awfully long time for an outage like this.

    We're currently chasing MS through the usual support channels, but having found nothing on google on this relating to GDPR data moves I thought I'd start something for anybody else experiencing this. I'm also not entirely sure this is in the correct Forum but couldn't find a more relevant one.


    • Edited by xhable Thursday, June 7, 2018 1:05 PM redacting the guid and formatting
    • Changed type Ed CrowleyMVP, Moderator Friday, August 31, 2018 4:11 PM Because it is
    Thursday, June 7, 2018 7:47 AM

All replies

  • First, please change your thread type to a question since you're asking one.

    Second, please fix your text so that it doesn't have imbedded HTML that makes it hard to read.

    When you've done that, I'll be happy to look at the question.


    Ed Crowley MVP "There are seldom good technological solutions to behavioral problems."
    Celebrating 20 years of providing Exchange peer support!

    Thursday, June 7, 2018 7:22 PM
    Moderator
  • same problem here occured just today
    Friday, August 31, 2018 2:53 PM
  • We are having the exact same issue, a Service that is using EWS (with no impersonation) simply does not go through. throws out the exact error in the service logs. Also connecting to office365 and using https://outlook.office365.com/ews/exchange.asmx as the Exchange server URL

    This issue has started with us on Tuesday in the afternoon.

    Friday, August 31, 2018 3:38 PM
  • The same error has started occurring for us as of yesterday Sept. 2nd


    • Edited by Mike Beaton Tuesday, September 4, 2018 10:00 AM us
    Monday, September 3, 2018 9:56 AM
  • We've got the same exception message since September 1st.
    Monday, September 3, 2018 10:44 AM
  • I believe that this is being looked at.

    Ed Crowley MVP "There are seldom good technological solutions to behavioral problems."
    Celebrating 20 years of providing Exchange peer support!

    Monday, September 3, 2018 9:45 PM
    Moderator
  • Dear Ed, Thanks for the update. Do you have any other source or links about all this? That would be really useful to get to know what's going on here! All of our infrastructure which depends on EWS - which is quite a lot of it - is down and has been since Sunday. I actually noticed that access to accounts disappeared progressively, one by one - which does fit in with the idea in xhable's first post above that this may be something to do with accounts moving - between servers? - behind the scenes, for whatever reason. Many thanks for your post and any further info!


    • Edited by Mike Beaton Tuesday, September 4, 2018 10:01 AM edit
    Tuesday, September 4, 2018 9:14 AM
  • I have nothing to report.

    Ed Crowley MVP "There are seldom good technological solutions to behavioral problems."
    Celebrating 20 years of providing Exchange peer support!

    Tuesday, September 4, 2018 3:41 PM
    Moderator
  • Dear @xhable , Did you get any further info, at all?
    Wednesday, September 5, 2018 10:19 AM
  • We are now getting this same error message - is there any update?
    Wednesday, September 5, 2018 10:31 AM
  • We've also now started encountering this error, and have been for 3 weeks now. We're running Codetwo's office 365 backup software, and have gone over the issues with their support many times, it all comes back to this same issue. Office365 not allowing access to certain mailboxes, at seemingly random intervals.
    Wednesday, September 5, 2018 10:49 AM
  • Hello.

    1. Please check status health Office 365 services on you Office 365 admin center.

    2. Please open ticket in Office 365 for your issue.

    3. Usually when you import big data to mailbox need check:

    3.1 License plan for mailbox.

    3.2 Limitation for import data. Exchange Online Limits  


    MCITP, MCSE. Regards, Oleg

    Wednesday, September 5, 2018 2:54 PM
  • Our 3rd party tech support company have been helping us to chase this up with MS. We now know that the problem occurs when the main mailbox and the delegated mailbox are on different Microsoft servers. Note that even when this is the case, it is known and acknowledged by MS that this only stops the main account from accessing the delegated account on EWS (not on Outlook desktop or Outlook 365 OWA).

    Btw, this fits with the idea in the OP that this has started occurring recently because of an MS data move of various mailboxes over to European servers. In our Exchange tenant account, we are using .uk.com and .com addresses (note that .uk.com is officially a UK domain, like .co.uk). (I am not yet 100% sure that this suggestion matches all our info, and will update...)

    Our tech support company have now run the fix of moving all the mailboxes which were still on the 'wrong' server (i.e. not the European one) across to the European one. This has worked and has resolved the problem, for us. I have not yet been filled in on the details of exactly what is required to do this, though I am pretty sure it is all done via Exchange PowerShell magic.




    • Edited by Mike Beaton Friday, September 7, 2018 2:10 PM update
    Wednesday, September 5, 2018 4:28 PM
  • We get sporadically the same error:

    The specified object was not found in the store., Can't connect to the mailbox of user Mailbox database guid: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently. ErrorProperties (0 items)

    Using managed EWS api to Office 365 public folder.

    Sometimes the error lasts for hours, sometimes for seconds.

    Powershell Get-MoveRequest Get-MoveRequestStatistics shows nothing.

    Microsoft Graph Api does not support public folders (yet or never?)

    Thursday, September 6, 2018 8:40 AM
  • @Meixger_IC You could try using Exchange PowerShell `Get-Mailbox` to check the server location for the two different mailboxes, and see if they are different? If so, this would match our version of the problem, so maybe our fix would fix it.

    The mailbox data field `MailboxLocations` seems to be the one with the relevant server info in it, from what I can see. As I say, I don't yet know the command to ask Exchange to move the box to a different server, since someone else was helping us to resolve this. (I'll post when I do, but probably a web search should turn it up too, I guess?)

    Good luck!





    • Edited by Mike Beaton Friday, September 7, 2018 2:15 PM update
    Friday, September 7, 2018 10:16 AM
  • We are experiencing the same issue as of September 14.

    Mailboxes are located in Europe. We are connected directly to a licensed mailbox.

    Everything was running fine for months and now we get the error.

    The specified object was not found in the store., Can't connect to the mailbox of user Mailbox database guid: because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently. 

    Microsoft technician has pointed me to the following link which is starting to explain things.

    https://support.microsoft.com/en-us/help/4133314/error-when-running-cmdlets-for-a-mailbox-in-another-region

    When forcing the connection to a Europe mailbox, powershell works. Now we just need to get EWS to work.

    Monday, September 17, 2018 3:53 PM
  • We are having the exact same issue, a Service that is using EWS (with no impersonation) simply does not go through. throws out the exact error in the service logs. Also connecting to office365 and using https://outlook.office365.com/ews/exchange.asmx as the Exchange server URL

    This issue has started with us on Tuesday in the afternoon.

    Microsoft Support suggested that I point to the link mentioned above and it works now. Sorry, I cannot post the link because apparently my account needs to be verified. It is the same as listed above though.


    Monday, September 17, 2018 7:19 PM
  • Please see this:

    https://blogs.msdn.microsoft.com/webdav_101/2016/04/12/ews-managed-api-coding-for-exchange/

    From the article:

    When EWS Impersonation is used the X-AnchorMailbox always should be correctly set.  Without doing so you may get 500 or 503 errors at times. It is critical for performance and also for notifications with Exchange Online/Exchange 2013.  Not setting it can double or more the time it takes to complete the call.  In some cases, you can also get timeouts.  The rule is to always set this header when using impersonation – this will make your EWS Impersonated code from Exchange 2007 work better with Exchange 2013.  It should be set to be set to the mailbox being accessed with the exception of when streaming notifications are being done and in that case it should be set to the first mailbox in the subscription group. With Exchange Online there are additional headers which need to be set for affinity.

    Example:  service.HttpHeaders.Add("X-AnchorMailbox", targetSmtp);

    Also see:

    How to: Route public folder hierarchy requests
    https://msdn.microsoft.com/en-us/library/office/dn818490(v=exchg.150).aspx

    Things to try if Impersonation is not working:

    • Be sure the credentials are correct.
    • Navigate to the EWS service URI in a browser and see if you get a log-in dialog box. This would likely be https://outlook.office365.com/EWS/Exchange.asmx
      for Exchange Online.
    • Try accessing the mailbox of the authenticating account using the owner's credentials using EWS and OWA.
    • Try accessing the mailbox being accessed using the its owner's credential using EWS and OWA.

    X-AnchorMailbox

    This header always should be set when using EWS Impersonation.  If you do not, then you may experience one of several issues which include slowness and errors such as a 500 or 403 error. When Impersonating is used, Exchange will be using the mailbox server of the impersonation account to move the calls through and as such can be slow if you don't tell it to use the server of the mailbox your accessing, which is done by using the X-AnchorMailbox header. Further, some calls may fail if you don't use this setting.

                   Example:  service.HttpHeaders.Add("X-AnchorMailbox", targetSmtp);


    Ed Crowley MVP "There are seldom good technological solutions to behavioral problems."
    Celebrating 20 years of providing Exchange peer support!

    Friday, September 21, 2018 5:00 AM
    Moderator
  • Hi All,

    Any update on this? we are getting the same issue since last week. No update from O365 support. Ticket is with product support team. Currently there is no SLA definition specific to Exchange Web services. Only SLAs are available for Exchange Online, Archiving and EOP. please provide your comments. Is EWS has same SLA as Exchange Online? 

    1. Public folders not visible in Microsoft 365 admin center, but can list through Exch online powershell

    2. Unable to access public folders through EWS.

    4. MS connectivity analyzer return the same EWS error which applications receive through EWS

    ErrorItemNotFound: The specified object was not found in the store., Can't connect to the mailbox of user Mailbox database guid: ddf2bf8f-2bcf-4e99-a01b-df1c608884f9 because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently.
    Elapsed Time: 481 ms.

    3.On navigating to in-place eDiscovery & hold, receiving below error

    error
    Can't connect to the mailbox of user Mailbox database guid: b7bc7d60-a6cf-4d9b-8423-a0de2245dd93 because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently.

    Comments pls. 

    Thursday, September 27, 2018 12:03 PM
  • My entire organization is sporadically experiencing this same error.  We've noticed it when viewing past mailbox searches, attempting to test a relay through Cisco Call Manager, and accessing shared mailboxes.  I have a ticket open with MS and requested it be included in what I'm sure is a parent ticket. 
    Thursday, September 27, 2018 6:10 PM
  • Same issues here with our 3rd party Document Management System.  I talked with a Microsoft Office 365 support manager after opening a ticket.  He claimed he never heard of this issue and stressed that it must be an issue with the 3rd party vendor.
    Friday, September 28, 2018 3:32 PM
  • See my last post above and share the information with your third-party document management vendor.

    Ed Crowley MVP "There are seldom good technological solutions to behavioral problems."
    Celebrating 20 years of providing Exchange peer support!

    Sunday, September 30, 2018 3:07 AM
    Moderator
  • This issue appears to be resolved for me now.  I can view mailbox searches using both Remote PowerShell and the EAC. 
    Friday, October 5, 2018 1:16 PM
  • Same on out tenant, not fixed.

    Thursday, October 11, 2018 6:30 PM
  • Seeing the same here. When Outlook for Mac users try to send mail from a shared mailbox, the messages don't send (they get stuck in Outbox).

    Also, when admins try to edit mailbox properties through EAC, they are receiving this error for some mailboxes:
    "Can't connect to the mailbox of user Mailbox database guid:{GUID} because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently."

    Microsoft Premier support has told us that our entire tenant is being moved to another forest. Due to this, in the scenario above, the user's mailbox and the shared mailbox are in different forests.

    If you run Get-Mailbox and look at the Database attribute, the first seven characters represent the forest.

    Due to the number of mailboxes on our tenant, it's going to take weeks before the move is done.

    For the time being, we're just initiating mailbox moves via PowerShell when a user reports that they can't send from a shared mailbox.

    Luckily, OWA and the Outlook client for Windows aren't affected by this.



    • Edited by Paul Kieper Thursday, October 18, 2018 4:08 PM
    Thursday, October 18, 2018 4:07 PM
  • We've been experiencing this issue for the past couple of months or so also (using an EOL hybrid environment). It occurs sporadically and doesn't always remain an issue (wait a couple hours or days and the problem resolves itself, until the next time it occurs).  We've been working with MS O365 for some time and (personally), I have my suspicions about the root cause.  Over the past couple of months, I've also seen intermittent problems when doing simple mailbox migrations (in our EOL hybrid environment).  The mailbox migration works but at the very end, all of a sudden the "outlook.com" domain controller cannot be contacted and the migration fails (FailedOther).  Not our on-premises AD but O365 Azure AD.  This is reminiscent of what we saw a few years back and to us, the problem was a resource availability issue at MS datacenters.  The last time these type of problems occurred, MS was "upgrading" the components of O365 to 2016.  Now, the issues are occurring again and we also just happen to be at that same point when MS will "start" upgrading O365 components to 2019.  We contacted them about it (last time) and MS denied it but we presented them with evidence and it turned out that the support staff who we (i.e. admins) talk to didn't know the upgrade processes were occurring (we had to escalate to the back-end teams).  Back then, coincidentally at the same time when MS was updating the O365 components to 2016, they were having resource issues (not enough resources in their datacenters) and they were scaling back and moving resources around (moving mailboxes, move AD, etc.).  We've started seeing EOL onboarding migrations stall again due to target disk latency the past couple of months (which is a MS datacenter SAN storage resource issue).  If you remember (years ago), we used to get lots of data on the admin portal dashboard but things like that were draining resources in O365, so MS redesigned the admin portal so that the resource drain didn't occur every time someone logs on to the admin portal by default.  These same types of issues occurred back then as well (domain services unavailable, mailboxes being moved around to free up resources, etc.).  Turns out that (at that time), they were onboarding a lot of government data (which put a drain on O365 resources even though the government systems are separated).  10-to-1 says that MS is either onboarding a very large tenant again and it is pulling resources away from the rest of us AND/OR that they are upgrading the O365 components from 2016 to 2019.  They slipped up last time (inadvertently, we started seeing things like our resource calendar notifications showing "Exchange 2016" at the bottom of the email notification before they announced the upgrades were occurring, stuff like that).  We also uncovered the URL's that showed the SPO component versions and we saw that it was being upgraded also (before the announcements were made that it would happen).  Chances are that it's either 2019 component upgrades (we're at that stage when the 2019 upgrades are rolling out) or they are shuffling resources around to onboard one or more large tenants to O365.  Thing is, the support staff that "we" get to speak to don't know what is happening behind-the-scenes from the back-end datacenter teams.  I get their business model perspective, but it impacts the rest of us poorly.

    Friday, October 19, 2018 8:58 AM
  • Seeing the same here. When Outlook for Mac users try to send mail from a shared mailbox, the messages don't send (they get stuck in Outbox).

    Also, when admins try to edit mailbox properties through EAC, they are receiving this error for some mailboxes:
    "Can't connect to the mailbox of user Mailbox database guid:{GUID} because the ExchangePrincipal object contains outdated information. The mailbox may have been moved recently."

    Microsoft Premier support has told us that our entire tenant is being moved to another forest. Due to this, in the scenario above, the user's mailbox and the shared mailbox are in different forests.

    If you run Get-Mailbox and look at the Database attribute, the first seven characters represent the forest.

    Due to the number of mailboxes on our tenant, it's going to take weeks before the move is done.

    For the time being, we're just initiating mailbox moves via PowerShell when a user reports that they can't send from a shared mailbox.

    Luckily, OWA and the Outlook client for Windows aren't affected by this.



    Hi Paul,

    Have exactly the same issue since 18 October, 2018. Can you please advice and help me to sort this out as I am total beginner of using 365 and powershell. I tried to run New-MoveRequest -Identity "mailname" and after Get-MoveRequestStatistics -Identity "mailname", and it shows that process is in Stall. Is there something else I need to do before I perform this action? Also should mention that I was trying to move shared mailbox as it is on different  Database from other two users using Macs.

    Monday, October 22, 2018 11:11 AM
  • ...

    3.2 Limitation for import data. Exchange Online Limits  


    MCITP, MCSE. Regards, Oleg

    The linked page contains no information about import data limitations. The word import does not appear on the page.
    Friday, October 26, 2018 3:34 PM
  • I'm facing this problem now.

    How do I solve this?

    Thank you

    Tuesday, November 20, 2018 12:13 PM
  • I'm facing this problem now.

    How do I solve this?

    Thank you

    Tuesday, November 20, 2018 12:14 PM
  • Hi Ray,

    did you manage to solve your issue? We are experiencing exactly the same error as of last week.

    Wednesday, April 10, 2019 10:00 AM
  • I've been having the same problem for the past 2 days. Everything has been fine for the last 3 years, then this. No changes to any setups.

    I have an image of the error but am not allowed to upload it.

    Ray, if you come upon a resolution, pleased share. Thanks.

    Tuesday, April 16, 2019 6:00 AM
  • How does my Account get verified so I may include a graphic in the body of my text?

    I don't see this written up any place.  Just the restriction.

    Tuesday, April 16, 2019 8:06 AM