none
Test-OAuth Fails from O365 to OnPrem. How to troubleshoot? RRS feed

  • Question

  • Check all client urls  are listed as HTTPS service principal names in AAD.

    Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 | select -ExpandProperty ServicePrincipalNames
    
    https://autodiscover.companyBCB46E.onelearndns.com/
    https://companyBCB46E.onelearndns.com/
    https://mailcompanybcb46e.onelearndns.com
    00000002-0000-0ff1-ce00-000000000000/mailcompanybcb46e.onelearndns.com
    00000002-0000-0ff1-ce00-000000000000/autodiscover.M365j823547.mail.onmicrosoft.com
    00000002-0000-0ff1-ce00-000000000000/M365j823547.mail.onmicrosoft.com
    00000002-0000-0ff1-ce00-000000000000/autodiscover.companyBCB46E.onelearndns.com
    00000002-0000-0ff1-ce00-000000000000/companyBCB46E.onelearndns.com
    00000002-0000-0ff1-ce00-000000000000/outlook.office365.com
    00000002-0000-0ff1-ce00-000000000000/mail.office365.com
    00000002-0000-0ff1-ce00-000000000000/outlook.com
    00000002-0000-0ff1-ce00-000000000000/*.outlook.com
    00000002-0000-0ff1-ce00-000000000000
    https://ps.compliance.protection.outlook.com
    https://autodiscover-s.office365.us/
    https://outlook.office365.us/
    https://outlook-sdf.office.com/
    https://outlook-sdf.office365.com/
    https://outlook.office365.com:443/
    https://outlook.office.com/
    https://outlook.office365.com/
    https://outlook.com/
    https://outlook-dod.office365.us/
    https://ps.protection.outlook.com/
    https://webmail.apps.mil/
    https://outlook-tdf.office.com/
    

    EvoSTS Auth Server Object is Present

    Get-AuthServer | where {$_.Name -eq "EvoSts"}
    
    
    Name   IssuerIdentifier                                              Realm                                TokenIssuingEndpoint                          Enabled
    ----   ----------------                                              -----                                --------------------                          -------
    EvoSts https://sts.windows.net/20d711ba-d28d-4756-8c59-1148086931a6/ 20d711ba-d28d-4756-8c59-1148086931a6 https://login.windows.net/common/oauth2/token True   
    
    
    

    Confirmed communication thru Netmon capture. Can see traffic come in all the way to Exchange, So doesn't appear to be network issue!!

    Test-OAuthConnectivity -Service EWS -TargetUri https://mailcompanybcb46e.onelearndns.com/metadata/json/1 -Mailbox eli@companybcb46e.onelearndns.com
    
    
    
    RunspaceId  : 7778e0a1-62a6-4f52-9eb3-889a674d8bd7
    Task        : Checking EWS API Call Under Oauth
    Detail      : The configuration was last successfully loaded at 1/1/0001 12:00:00 AM UTC. This was 1062143631 minutes 
                  ago.
                  The token cache is being cleared because "use cached token" was set to false.
                  Values for header request-id:
                  	03526bfd-603d-4791-911a-7f7c1dfeaa30Values for header Cache-Control:
                  	privateValues for header Content-Type:
                  	text/xml; charset=utf-8Values for header Set-Cookie:
                  	X-BackEndCookie=actas1(sip:eli@companyBCB46E.onelearndns.com|smtp:eli@companyBCB46E.onelearndns.com|upn:
                  eli@companyBCB46E.onelearndns.com)=u56Lnp2ejJqBzs3OnMnLysvSy8fPxtLLnsue0p7Iz8nSypzImcrLz8jOmpzOgYHNz83P0s
                  /I0s3Lq87IxcrPxcvPgY+ZmpOenYzRk5CcnpOBzg==; Expires=Fri, 24-Jul-2020 17:50:40 GMT; Path=/ews; Secure; 
                  HttpOnly	exchangecookie=1c341fa52fc546bea5c831f88ee260e8; Expires=Thu, 24-Jun-2021 17:50:40 GMT; Path=/; 
                  HttpOnlyValues for header Server:
                  	Microsoft-IIS/10.0 Microsoft-HTTPAPI/2.0Values for header x-aspnet-version:
                  	4.0.30319Values for header x-beserver:
                  	E2K19Values for header x-calculatedbetarget:
                  	e2k19.pfelabs.localValues for header x-diaginfo:
                  	E2K19Values for header x-feserver:
                  	E2K19Values for header x-powered-by:
                  	ASP.NETValues for header Date:
                  	Wed, 24 Jun 2020 17:50:40 GMTExchange Outbound Oauth Log:
                  Client request ID: 5f7a3466-3f1d-4624-8219-23df78c2e94d
                  Information:[OAuthCredentials:Authenticate] entering
                  Information:[OAuthCredentials:Authenticate] challenge from 
                  'https://mailcompanybcb46e.onelearndns.com/ews/Exchange.asmx' received: Negotiate,NTLM,Bearer 
                  client_id="00000002-0000-0ff1-ce00-000000000000", 
                  trusted_issuers="00000001-0000-0000-c000-000000000000@20d711ba-d28d-4756-8c59-1148086931a6", 
                  token_types="app_asserted_user_v1 service_asserted_app_v1" 
                  Information:[OAuthCredentials:GetToken] client-id: '00000002-0000-0ff1-ce00-000000000000', realm: '', 
                  trusted_issuer: '00000001-0000-0000-c000-000000000000@20d711ba-d28d-4756-8c59-1148086931a6'
                  Information:[OAuthCredentials:GetToken] Start building a token using organizationId 
                  '20d711ba-d28d-4756-8c59-1148086931a6'
                  Information:[OAuthTokenBuilder:GetAppToken] start building the apptoken
                  Information:[OAuthTokenBuilder:GetAppToken] checking enabled auth servers
                  Information:[OAuthTokenBuilder:GetAppToken] trusted_issuer includes the auth server 'Sts': 
                  00000001-0000-0000-c000-000000000000@*, 
                  Information:[OAuthTokenBuilder:GetAppToken] trying to get the apptoken from the auth server 'Sts' for 
                  resource '00000002-0000-0ff1-ce00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-
                  1148086931a6', tenantId '20d711ba-d28d-4756-8c59-1148086931a6', userDomain 
                  'companyBCB46E.onelearndns.com'
                  Information:[TokenCache:GetActorToken] Each key and its counts are 
                  L:00000002-0000-0ff1-ce00-000000000000-AS:00000001-0000-0000-c000-000000000000@*, 0
                  Information:[TokenCache:GetActorToken] cache size is 0
                  Information:[TokenCache:GetActorToken] try to get a new  token synchronously
                  Information:[ACSTokenBuildRequest:BuildToken] started
                  Information:[TokenBuildRequest:GetActorTokenFromAuthServer] Sending token request to 
                  'https://accounts.accesscontrol.windows.net/tokens/OAuth/2' for the resource '00000002-0000-0ff1-ce00-000
                  000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6' with token: grant_type=
                  http%3a%2f%2foauth.net%2fgrant_type%2fjwt%2f1.0%2fbearer&assertion=eyJhbGciOiJSUzI1NiIsImtpZCI6IjU2MzU4OD
                  UyMzRCOTI1MkRERTAwNTc2NkQ5RDlGMjc2NTY1RjYzRTIiLCJ4NXQiOiJWaldJVWpTNUpTM2VBRmRtMmRueWRsWmZZLUkiLCJ0eXAiOiJ
                  KV1QifQ.eyJuYmYiOjE1OTMwMjEwNDAsImV4cCI6MTU5MzAyMTY0MCwiaXNzIjoiMDAwMDAwMDItMDAwMC0wZmYxLWNlMDAtMDAwMDAwM
                  DAwMDAwQDIwZDcxMWJhLWQyOGQtNDc1Ni04YzU5LTExNDgwODY5MzFhNiIsImF1ZCI6IjAwMDAwMDAxLTAwMDAtMDAwMC1jMDAwLTAwMD
                  AwMDAwMDAwMC9hY2NvdW50cy5hY2Nlc3Njb250cm9sLndpbmRvd3MubmV0QDIwZDcxMWJhLWQyOGQtNDc1Ni04YzU5LTExNDgwODY5MzF
                  hNiJ9.PwdAT_zJV3wlVuglU7lGUoxE7OSqQVPZmes7myPXOlOciotPKOJlCpirsKFB5neYEgjudiFGlPM23cbKOH2jSGTIz8mkDbcnUSR
                  qYOivPupNAGhuf2sP5FK84-h3xjYzluWQI1eVdW0FYnS9F-09Wzfq9muGDghyKKxWJoqcB4ZHI-0-nI5b5nsylPwYSDZ8oTvYmrlxmm6i
                  5rGXPiyHSrLVZkkNgmJM72fPBl-NIEFPCaz2-ehUczEjna_dhc_ghvtMzxTZOgQJuprNM1kQxRvSegfIHkIIPjmzjEeG5gqfKkteA55JF
                  JiWSyC8oaAIPe-0U2tgE78HmSgw46BjXQ&resource=00000002-0000-0ff1-ce00-000000000000%2fmailcompanybcb46e.onele
                  arndns.com%4020d711ba-d28d-4756-8c59-1148086931a6
                  Information:[TokenBuildRequest:GetActorTokenFromAuthServer] response headers was 
                  Pragma: no-cache
                  Strict-Transport-Security: max-age=31536000; includeSubDomains
                  X-Content-Type-Options: nosniff
                  client-request-id: 5f7a3466-3f1d-4624-8219-23df78c2e94d
                  x-ms-request-id: 843d39b5-ffcc-4bc9-86bf-1d8e13e40801
                  x-ms-ests-server: 2.1.10732.8 - EST ProdSlices
                  Cache-Control: no-cache, no-store
                  Content-Type: application/json; charset=utf-8
                  Expires: -1
                  P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
                  Set-Cookie: fpc=AswfKf1V3VRLnS1_tMZLTTDQNUiqAQAAAG-JhdYOAAAA; expires=Fri, 24-Jul-2020 17:50:40 GMT; 
                  path=/; secure; HttpOnly; SameSite=None,esctx=AQABAAAAAAAm-06blBE1TpVMil8KPQ41H4tX6d3glSB7Dh4nuAdhkg15CoJ
                  z3VbC015fJ7YOjZ_y-BFXEtWYVTL8citcaG-3OXuEj0sDkl78asdlGuzK4KsnnHTFdjrNew8yGguVSAiORdhnFNjZQn9FsCuIdK1bvg4q
                  B52Cc4TddoeQiaQurhUVCameAH6PVb6RZmr4L3wgAA; domain=.accounts.accesscontrol.windows.net; path=/; secure; 
                  HttpOnly; SameSite=None,x-ms-gateway-slice=prod; path=/; secure; HttpOnly,stsservicecookie=ests; path=/; 
                  secure; HttpOnly
                  Date: Wed, 24 Jun 2020 17:50:40 GMT
                  Content-Length: 1445
                  
                  
                  Information:[ACSTokenBuildRequest:BuildToken] finished
                  Information:[OAuthTokenBuilder:GetAppToken] finish building apptoken; the token is {"typ":"JWT","alg":"RS
                  256","x5t":"SsZsBNhZcF3Q9S4trpQBTByNRRI","kid":"SsZsBNhZcF3Q9S4trpQBTByNRRI"}."oid": 
                  "a1ce4129-026c-4b9f-83b9-2d1bc4dcb929" "iss": 
                  "00000001-0000-0000-c000-000000000000@20d711ba-d28d-4756-8c59-1148086931a6" "aud": "00000002-0000-0ff1-ce
                  00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6" "nbf": 
                  "1593020740" "exp": "1593107440" 
                  Information:[OAuthTokenBuilder.GetAppWithUserToken] nameid is allowed to be included in the claim set
                  Information:[OAuthTokenBuilder.GetAppWithUserToken] only nameid to be included in the claim: no
                  Information:[OAuthTokenBuilder.GetAppWithUserToken] building token with user context for the audience '00
                  000002-0000-0ff1-ce00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6
                  '
                  Information:[OAuthCredentials:Authenticate] send request to 
                  'https://mailcompanybcb46e.onelearndns.com/ews/Exchange.asmx' with the bearer token: 
                  '{"alg":"none","typ":"JWT"}."iss": 
                  "00000002-0000-0ff1-ce00-000000000000@20d711ba-d28d-4756-8c59-1148086931a6" "aud": "00000002-0000-0ff1-ce
                  00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6" "nbf": 
                  "1593021040" "exp": "1593049840" ; actor: {"typ":"JWT","alg":"RS256","x5t":"SsZsBNhZcF3Q9S4trpQBTByNRRI",
                  "kid":"SsZsBNhZcF3Q9S4trpQBTByNRRI"}."oid": "a1ce4129-026c-4b9f-83b9-2d1bc4dcb929" "iss": 
                  "00000001-0000-0000-c000-000000000000@20d711ba-d28d-4756-8c59-1148086931a6" "aud": "00000002-0000-0ff1-ce
                  00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6" "nbf": 
                  "1593020740" "exp": "1593107440" '
                  Token:{"alg":"none","typ":"JWT"}."iss": 
                  "00000002-0000-0ff1-ce00-000000000000@20d711ba-d28d-4756-8c59-1148086931a6" "aud": "00000002-0000-0ff1-ce
                  00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6" "nbf": 
                  "1593021040" "exp": "1593049840" ; actor: {"typ":"JWT","alg":"RS256","x5t":"SsZsBNhZcF3Q9S4trpQBTByNRRI",
                  "kid":"SsZsBNhZcF3Q9S4trpQBTByNRRI"}."oid": "a1ce4129-026c-4b9f-83b9-2d1bc4dcb929" "iss": 
                  "00000001-0000-0000-c000-000000000000@20d711ba-d28d-4756-8c59-1148086931a6" "aud": "00000002-0000-0ff1-ce
                  00-000000000000/mailcompanybcb46e.onelearndns.com@20d711ba-d28d-4756-8c59-1148086931a6" "nbf": 
                  "1593020740" "exp": "1593107440" 
                  
                  Exchange Response Details:
                  HTTP response message: 
                  Exception:
                  System.IO.IOException: Unable to read data from the transport connection: An existing connection was 
                  forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was 
                  forcibly closed by the remote host
                     at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
                     --- End of inner exception stack trace ---
                     at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
                     at System.IO.StreamReader.ReadBuffer()
                     at System.IO.StreamReader.ReadToEnd()
                     at Microsoft.Exchange.Monitoring.TestOAuthConnectivityHelper.SendExchangeOAuthRequest(ADUser user, 
                  String orgDomain, Uri targetUri, String& diagnosticMessage, Boolean appOnly, Boolean useCachedToken, 
                  Boolean reloadConfig)
                  
    ResultType  : Error
    Identity    : Microsoft.Exchange.Security.OAuth.ValidationResultNodeId
    IsValid     : True
    ObjectState : New
    
    
    
    



    With Regards, M S Ali

    Wednesday, June 24, 2020 6:16 PM

All replies

  • Hi MS Ali,

    From the information below, we can know that this issue is more related with firewall. Even though you have see some request has come to your Exchange server, but, some of them may be blocked:

    Since this issue is more related with firewall, I would suggest you try to temporarily disable intermediate equipment. If this issue gone, it mean this issue caused by firewall or other intermediate equipment, you can try to add all Office 365 IP to the whitelist on your intermediate equipment.

    Regards,

    Kyle Xu


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Thursday, June 25, 2020 5:46 AM
    Moderator
  • Hi! Kyle,

    That was our first impression too and standard approach in these kind of error. But I traced the incoming request from the reverse proxy all the way to exchange server so I know that all packet reaching the reverse proxy is making to exchange server. Validated with NetMon also. I think the forcible closed connection is because of Exchange closing the connection I can see an ACK RST packet being send back in the end. In the httpproxy log I see repeated 401/200 indicative of not able to authenticate and think after some time the exchange just closed the connection as it was not able to negotiate authentication. 

    We did further inspection of the HTTPS packet and saw that in response to the user availability request it is sending an exception back with Error code 5027. Have you come across this Exception Error Code? 

    Is there anyway we can configure Exchange Server 2019 to use DAUTH for Free/Busy instead of OAUTH  using traditional Federation Trust?



    With Regards, M S Ali

    Thursday, June 25, 2020 12:56 PM
  • Hi! Kyle,

    That was our first impression too and standard approach in these kind of error. But I traced the incoming request from the reverse proxy all the way to exchange server so I know that all packet reaching the reverse proxy is making to exchange server. Validated with NetMon also. I think the forcible closed connection is because of Exchange closing the connection I can see an ACK RST packet being send back in the end. In the httpproxy log I see repeated 401/200 indicative of not able to authenticate and think after some time the exchange just closed the connection as it was not able to negotiate authentication. 

    We did further inspection of the HTTPS packet and saw that in response to the user availability request it is sending an exception back with Error code 5027. Have you come across this Exception Error Code? 

    Is there anyway we can configure Exchange Server 2019 to use DAUTH for Free/Busy instead of OAUTH  using traditional Federation Trust?



    With Regards, M S Ali

    I don't find information about DAUTH. The supported way is to using OAUTH.  

    Regards, 

    Kyle Xu


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Tuesday, June 30, 2020 9:05 AM
    Moderator
  • Hi MS Ali,

    Any update about this thread now?

    Regards,

    Kyle Xu


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Friday, July 3, 2020 6:30 AM
    Moderator