none
Exchange 2019 CU2 breaks AD FS integration with ECP and OWA RRS feed

  • Question

  • Hello all,

    After updating from CU1 to CU2, attempting to access OWA and ECP after successfully logging in to ADFS fails and returns an error 500. The following event is logged on the Exchange server:

    [Owa] An internal server error occurred. The unhandled exception was: System.InvalidCastException: Unable to cast object of type 'Microsoft.Exchange.Security.Authentication.AdfsIdentity' to type 'System.Security.Principal.WindowsIdentity'.
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.ReadADUser(String userKey, HttpApplication httpApplication, IRecipientSession recipientSession)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.LoadUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, RootOrgContainerIdWrapper rootOrgWrapper)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyRepo.GetUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyEvaluator.IsBasicAuthAllowed(String userKey, String protocolName, Int32 traceId, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.HttpProxy.ProxyModule.IsLegacyAuthAllowed(HttpApplication httpApplication)
       at Microsoft.Exchange.HttpProxy.ProxyModule.OnPostAuthenticateInternal(HttpApplication httpApplication)
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)

    I also see the following .NET warning:

    Event code: 3005
    Event message: An unhandled exception has occurred.
    Event time: 7/5/2019 4:59:44 PM
    Event time (UTC): 7/5/2019 11:59:44 PM
    Event ID: 212fc75cc52a4517b9ac686b8c432517
    Event sequence: 2
    Event occurrence: 1
    Event detail code: 0
     
    Application information:
        Application domain: /LM/W3SVC/1/ROOT/ecp-1-132068447039404428
        Trust level: Full
        Application Virtual Path: /ecp
        Application Path: C:\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\ecp\
        Machine name: EX2019MB1
     
    Process information:
        Process ID: 16264
        Process name: w3wp.exe
        Account name: NT AUTHORITY\SYSTEM
     
    Exception information:
        Exception type: InvalidCastException
        Exception message: Unable to cast object of type 'Microsoft.Exchange.Security.Authentication.AdfsIdentity' to type 'System.Security.Principal.WindowsIdentity'.
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.ReadADUser(String userKey, HttpApplication httpApplication, IRecipientSession recipientSession)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.LoadUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, RootOrgContainerIdWrapper rootOrgWrapper)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyRepo.GetUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyEvaluator.IsBasicAuthAllowed(String userKey, String protocolName, Int32 traceId, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.HttpProxy.ProxyModule.IsLegacyAuthAllowed(HttpApplication httpApplication)
       at Microsoft.Exchange.HttpProxy.ProxyModule.OnPostAuthenticateInternal(HttpApplication httpApplication)
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
       at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
       at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
       at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

     
     
    Request information:
        Request URL: https://mail.ambientitsolutions.com:443/ecp/
        Request path: /ecp/
        User host address: 192.168.200.112
        User: S-1-5-21-2852231475-3427835444-4095336066-10110
        Is authenticated: True
        Authentication Type: ADFS
        Thread account name: NT AUTHORITY\SYSTEM
     
    Thread information:
        Thread ID: 63
        Thread account name: NT AUTHORITY\SYSTEM
        Is impersonating: False
        Stack trace:    at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.ReadADUser(String userKey, HttpApplication httpApplication, IRecipientSession recipientSession)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyLoader.LoadUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, RootOrgContainerIdWrapper rootOrgWrapper)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyRepo.GetUserPolicy(String userKey, Int32 traceId, Int32& userPolicy, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.Security.Authentication.FederatedAuthService.BasicAuthPolicyEvaluator.IsBasicAuthAllowed(String userKey, String protocolName, Int32 traceId, HttpApplication httpApplication, IRecipientSession recipientSession, IConfigurationSession configSession, ConfigWrapper config)
       at Microsoft.Exchange.HttpProxy.ProxyModule.IsLegacyAuthAllowed(HttpApplication httpApplication)
       at Microsoft.Exchange.HttpProxy.ProxyModule.OnPostAuthenticateInternal(HttpApplication httpApplication)
       at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
       at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
       at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
       at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
     
     
    Custom event details:

    I've restarted the server, disabled and re-enabled AD FS authentication on the virtual directory (and iisreset after each of these), but the problem still occurs. I unfortunately have not found reports of a similar exception on the Internet. Any ideas?

    Saturday, July 6, 2019 12:01 AM

Answers

All replies

  • Hi Ckindley,

    Please run the following command to check the authentication method on OWA and ECP virtual directories:

    Get-EcpVirtualDirectory |fl *authentication*
    Get-EcpVirtualDirectory |fl *authentication*
    Make sure that the ADFS authentication as the only authentication method is enabled and all other forms of authentication are disabled.


    Best Regards,
    Niko Cheng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Monday, July 8, 2019 9:27 AM
    Moderator
  • Hello Niko,

    Thanks for the reply. Here's the output of that cmdlet:

    InternalAuthenticationMethods : {Adfs}
    BasicAuthentication           : False
    WindowsAuthentication         : False
    DigestAuthentication          : False
    FormsAuthentication           : False
    LiveIdAuthentication          : False
    AdfsAuthentication            : True
    OAuthAuthentication           : False
    ExternalAuthenticationMethods : {Fba}

    InternalAuthenticationMethods : {Adfs}
    BasicAuthentication           : False
    WindowsAuthentication         : False
    DigestAuthentication          : False
    FormsAuthentication           : False
    LiveIdAuthentication          : False
    AdfsAuthentication            : True
    OAuthAuthentication           : False
    ExternalAuthenticationMethods : {Fba}

    The output of

    Get-OwaVirtualDirectory | fl *authentication*

    is the same. Based on the results, it looks like the non-working (first set) server is configured identically to the working (second set) server. This is what I expected, as I re-ran the cmdlets to enable only ADFS after completing the install and finding the problem. Any other ideas?

    Thanks!

    Monday, July 8, 2019 4:46 PM
  • On my Exchange 2019 CU2 the same problem. I communicate with Microsoft support. The support engineer thinks the problem is on the ADFS side, but it's not proven.
    Tuesday, July 9, 2019 12:07 PM
  • Thanks for the response! I'm on the verge of opening a partner case, but I will poke around AD FS first. Definitely update this thread if you come across the resolution.
    Tuesday, July 9, 2019 3:22 PM
  • Still broken after installing the July Windows Updates on all servers involved. Save us, Microsoft!
    Wednesday, July 10, 2019 11:33 PM
  • Hi ckindley,

    You can try to re-configure ADFS authentication as a workaround and check if any helps:

    Configure the Exchange organization to use AD FS authentication

    If still does not work, I'd recommend you open a ticket to help you dig in this issue deeply.


    Best Regards,
    Niko Cheng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.


    Friday, July 12, 2019 7:58 AM
    Moderator
  • Hi Niko,

    Thanks for the advice. I actually hadn't re-configured AD FS integration. Ran through that again, but the problem persists. I'll have to open a case next week to dig in - today is for a SCSM case! I'll update if we figure out anything generalizable.

    Carter

    Friday, July 12, 2019 5:18 PM
  • Hi Carter,

    I get a notification today, it states that this is a known issue in Exchange 2019 CU2 and the engineering team has been aware of this problem, this issue would be fixed in next CU (Exchange 2019 CU3). At present, the workaround is to disable ADFS on the Exchange 2019 virtual directories. 

    Thanks for your understanding, 


    Best Regards,
    Niko Cheng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.


    Monday, July 15, 2019 2:15 AM
    Moderator
  • This should really be documented as a known issue or the CU pulled entirely. Disabling ADFS is not an option for many people due to other dependencies (two factor auth, etc).
    Monday, July 15, 2019 3:25 PM
  • Niko,

    Thanks for following up on this thread. Such a severe problem should result in a recall of the CU and correction/reissuance in a short period of time. Hope CU3 comes soon.

    Mark49808,

    Agreed, and that's the case for us. Fortunately I have one that I did not install CU2 on and can balance requests to OWA and ECP to that. Should be able to skate by until CU3, but it's not an ideal situation.

    Carter

    Monday, July 15, 2019 4:28 PM
  • Another workaround is to keep some servers on CU1 and route incoming requests to those. We are also updating the KB with this as a known issue. 
    Monday, July 15, 2019 6:11 PM
  • What KB are you updating with the known issues? i'd like a link to that.
    Friday, July 19, 2019 8:55 PM
  • Do you have a link to download ExchangeServer2019-x64-cu1.iso?

    I tried VLSC but unfortunately only CU2 resides there....

    Wednesday, August 28, 2019 11:33 AM
  • Give CSS a call and request the bits. 
    Wednesday, August 28, 2019 8:16 PM
  • Any progress on your issue with ADFS and Exchange 2019? This seems like a concerning bug, I am kind of holding off on updating if ADFS OWA/ECP Integration does not play nice with EX2019.
    Wednesday, September 4, 2019 1:40 AM
  • The issue is resolved in CU3 which is just a few weeks away. 
    Wednesday, September 4, 2019 1:43 AM
  • That is good to know, Greg.

    Do you have a source for this?

    Wednesday, September 4, 2019 1:49 AM
  • Do you mean, who told me? 
    Wednesday, September 4, 2019 1:50 AM
  • Yes, was this something you opened a ticket for with MS and they confirmed the fix to your issue will be coming out in CU3?
    Wednesday, September 4, 2019 1:52 AM
  • I'm the Director of Marketing for Exchange. I know a few people. 
    Wednesday, September 4, 2019 1:56 AM
  • That is good enough for me, thanks for confirming that the fix will be out in CU3.
    Wednesday, September 4, 2019 1:57 AM
  • Greg, Can you confirm ADFS integration is fixed in CU3 (just released)? Its nowhere to be found in the release notes.

    https://support.microsoft.com/en-us/help/4514141/cumulative-update-3-for-exchange-server-2019

    Tuesday, September 17, 2019 5:43 PM
  • I would like to know this as well, part of the reason why we're holding off on upgrading to latest Exchange is this ADFS integration bug.
    Tuesday, September 17, 2019 6:04 PM
  • I installed in test and it seems to work with ADFS. But someone else confirming would be appreciated.
    • Proposed as answer by fastBONE Wednesday, September 18, 2019 4:29 AM
    • Unproposed as answer by fastBONE Wednesday, September 18, 2019 4:29 AM
    Tuesday, September 17, 2019 7:52 PM
  • I can confirm the fix is in the build - the lack of KB was a mistake being rectified now. It will be added to the fix list KB in the next few days. 
    • Proposed as answer by fastBONE Wednesday, September 18, 2019 4:29 AM
    Wednesday, September 18, 2019 12:50 AM
  • I installed in test and it seems to work with ADFS. But someone else confirming would be appreciated.


    I can confirm it's okay now!
    Wednesday, September 18, 2019 4:53 AM
  • I installed in test and it seems to work with ADFS. But someone else confirming would be appreciated.


    I can confirm it's okay now!

    I can also confirm that Using ADFS for authentication works with Exchange 2019 CU3

    \o/


    Please remember to mark my replies as answers if they help

    Thursday, September 19, 2019 4:26 PM
  • Not to hijack this thread, but Greg Taylor since you are Director of Marketing for Exchange - there is another thread on here requiring some serious attention from an official source, Search bug with Exchange 2019.

    Can you take a look and comment within that thread? Given you reach within the Exchange team it would be appreciated. There is a growing concern related to a search bug in EX2019 and it has not been addressed in CU3.


    • Edited by techy86_ Friday, September 20, 2019 3:17 PM
    Friday, September 20, 2019 3:14 PM
  • Thanks for the heads up. I'll have a look. 
    Friday, September 20, 2019 3:15 PM
  • Thanks for the quick reply :)
    Friday, September 20, 2019 3:18 PM