locked
ADFS (2016) Farm: problem with Rich Client Application using second ADFS Server RRS feed

  • Question

  • Hello,

    I have a ADFS farm with 2 servers based on Windows Server 2016 + WAP Farm based on Windows Server 2016 with NLB.
    There couple published applications that use ADFS preauthentication on the WAP farm. One of them - HTTP Basic for Rich clients application,
     we use it for active sync.

     ADFS farm uses WID database and include two servers: adfs-01.bfds.local and adfs-02.bfds.local.The shared name of ADFS farm is adfs.company.com (external)
     There no NLB servers in front of them, I switch between them manually in DNS (changing the IP address in A record of adfs farm shared name).

    The problem is:

     Everything works fine with adfs-01 server.
     When I'm switching to the adfs-02 server, standard published applications (Web and MSOFBA in WAP terminology) continue to work without the problem, but Active Sync application (HTTP Basic for Rich clients applications in WAP terminology) works only till the moment when the token (issued with adfs-01 server) becomes inactive.

     Some details about infrastructure:
     WAP servers (in DMZ): 192.168.210.41 (wap-01) and 192.168.210.42 (wap-02)
     ADFS server: 192.168.200.191 (adfs-01.bfds.local), 192.168.200.192 (adfs-02.bfds.local)
     ADFS farm name: adfs.company.com
     kb4041688 installed on ADFS servers

     I can't find anything bad in event logs on WAP and ADFS servers. I tried to get trace logs on ADFS-02 server and found this (when the problem happens):

    Info:

    Received request with following properties: 
    
    Date: 2017-12-07 13:12:15
    Remote endpoint: 192.168.210.42
    Local endpoint: 192.168.200.192
    Http method: POST
    Request Url: /adfs/proxy/relyingpartytoken
    Query string: ?api-version=1
    Local Port: 443
    User agent string: -
    Body data length: 591
    Caller Identity: -
    Certificate Identity: -
    Relying Party: -
    Through proxy: False
    Proxy name: -
    Serialized Header: {"Connection":"Keep-Alive","Content-Length":"591","Host":"adfs.company.com","X-MS-Endpoint-Absolute-Path":"/adfs/proxy/relyingpartytoken"}
    

    Info:

    Following request context headers present: 
    
    X-MS-Client-Application: -
    X-MS-Client-User-Agent: -
    client-request-id: -
    X-MS-Endpoint-Absolute-Path: /adfs/proxy/relyingpartytoken
    X-MS-Forwarded-Client-IP: -
    X-MS-Proxy: -
    X-MS-ADFS-Proxy-Client-IP: -

    Verbose:

    ContextCleanup: Cleaning up diagnostics and request state on the thread.

    Verbose:

    ContextCleanup: Cleaning up diagnostics and request state on the thread.

    Verbose:

    SetDiagnosticsInfoFromIncomingMessage: clientRequestId either not present in querystring or http header or is not a valid guid, so using one of our own: 27e1e9ce-5999-421e-b007-0080000000f6

    Verbose:

    Request does not contain 'return-client-request-id' header so not adding 'client-request-id' header to the response

    Info:

    ProxyConfigurationListener.OnGetContext: RequestHandler chosen: Microsoft.IdentityServer.Web.Proxy.RelyingPartyTokenHandler

    Error:

    ProxyServerRequestHandler.ProcessContext: WebException returned : Exception: The remote server returned an error: (503) Server Unavailable.
    StackTrace:    at System.Net.HttpWebRequest.GetResponse()
       at Microsoft.IdentityServer.Web.Proxy.PrimaryServerRequestHandler.ProcessContext(WrappedHttpListenerContext context, String authenticationAuditInformation)

    Info:

       Sending response at time: '2017-12-07 13:12:15' with StatusCode: '503' and StatusDescription: 'Service Unavailable'.
    Response headers set: {"Content-Type":"text/html; charset=utf-8"} 

    And in the security log a lot of:

    The Federation Service failed to issue a valid token. See XML for failure details. 
    
    Activity ID: 848a4f7d-f8cd-48bf-a202-0080010000df 
    
    Additional Data 
    XML: <?xml version="1.0" encoding="utf-16"?>
    <AuditBase xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AppTokenAudit">
      <AuditType>AppToken</AuditType>
      <AuditResult>Failure</AuditResult>
      <FailureType>GenericError</FailureType>
      <ErrorCode>N/A</ErrorCode>
      <ContextComponents>
        <Component xsi:type="ResourceAuditComponent">
          <RelyingParty>N/A</RelyingParty>
          <ClaimsProvider>N/A</ClaimsProvider>
          <UserId>N/A</UserId>
        </Component>
        <Component xsi:type="AuthNAuditComponent">
          <PrimaryAuth>N/A</PrimaryAuth>
          <DeviceAuth>false</DeviceAuth>
          <DeviceId>N/A</DeviceId>
          <MfaPerformed>false</MfaPerformed>
          <MfaMethod>N/A</MfaMethod>
          <TokenBindingProvidedId>false</TokenBindingProvidedId>
          <TokenBindingReferredId>false</TokenBindingReferredId>
          <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
        </Component>
        <Component xsi:type="ProtocolAuditComponent">
          <OAuthClientId>N/A</OAuthClientId>
          <OAuthGrant>N/A</OAuthGrant>
        </Component>
        <Component xsi:type="RequestAuditComponent">
          <Server>N/A</Server>
          <AuthProtocol>N/A</AuthProtocol>
          <NetworkLocation>Intranet</NetworkLocation>
          <IpAddress>192.168.210.42</IpAddress>
          <ProxyServer>N/A</ProxyServer>
          <UserAgentString>N/A</UserAgentString>
          <Endpoint>/adfs/proxy/relyingpartytoken</Endpoint>
        </Component>
      </ContextComponents>
    </AuditBase>

     Steps to reproduce the problem:
     1. Switch A record of ADFS farm shared name (adfs.company.com) to the adfs-02 server IP in DNS.
     2. Restart WAP services to flush all tokens in memory.

    ------------

    Test-AdfsServerHealth on primary ADFS (adfs-01.bfds.local):

    Name                                                       Result
    ----                                                       ------
    IsAdfsRunning                                                Pass
    IsWidRunning                                                 Pass
    PingFederationMetadata                                       Pass
    CheckAdfsSslBindings                                         Pass
    Test-Certificate-Token-Decrypting-Primary-NotFoundInStore  NotRun
    Test-Certificate-Token-Decrypting-Primary-IsSelfSigned     NotRun
    Test-Certificate-Token-Decrypting-Primary-PrivateKeyAbsent NotRun
    Test-Certificate-Token-Decrypting-Primary-Expired            Pass
    Test-Certificate-Token-Decrypting-Primary-Revoked            Pass
    Test-Certificate-Token-Decrypting-Primary-AboutToExpire    NotRun
    Test-Certificate-Token-Signing-Primary-NotFoundInStore     NotRun
    Test-Certificate-Token-Signing-Primary-IsSelfSigned        NotRun
    Test-Certificate-Token-Signing-Primary-PrivateKeyAbsent    NotRun
    Test-Certificate-Token-Signing-Primary-Expired               Pass
    Test-Certificate-Token-Signing-Primary-Revoked               Pass
    Test-Certificate-Token-Signing-Primary-AboutToExpire       NotRun
    Test-Certificate-SSL-Primary-NotFoundInStore                 Pass
    Test-Certificate-SSL-Primary-IsSelfSigned                    Pass
    Test-Certificate-SSL-Primary-PrivateKeyAbsent                Pass
    Test-Certificate-SSL-Primary-Expired                         Pass
    Test-Certificate-SSL-Primary-Revoked                         Pass
    Test-Certificate-SSL-Primary-AboutToExpire                   Pass
    CheckFarmDNSHostResolution                                   Pass
    CheckDuplicateSPN                                            Pass
    TestServiceAccountProperties                                 Pass
    TestAppPoolIDMatchesServiceID                              NotRun
    TestComputerNameEqFarmName                                   Pass
    TestSSLUsingADFSPort                                       NotRun
    TestSSLCertSubjectContainsADFSFarmName                       Pass
    TestAdfsAuditPolicyEnabled                                   Pass
    TestAdfsRequestToken                                         Pass
    CheckOffice365Endpoints                                      Pass
    TestADFSO365RelyingParty                                   NotRun
    TestNtlmOnlySupportedClientAtProxyEnabled                    Fail

    Test-AdfsServerHealth on second ADFS (adfs-02.bfds.local):

    Name                                                             Result
    ----                                                             ------
    IsAdfsRunning                                                      Pass
    IsWidRunning                                                       Pass
    PingFederationMetadata                                             Pass
    CheckAdfsSslBindings                                             NotRun
    Test-Certificate-Service-Communications-Primary-NotFoundInStore  NotRun
    Test-Certificate-Service-Communications-Primary-IsSelfSigned     NotRun
    Test-Certificate-Service-Communications-Primary-PrivateKeyAbsent NotRun
    Test-Certificate-Service-Communications-Primary-Expired          NotRun
    Test-Certificate-Service-Communications-Primary-Revoked          NotRun
    Test-Certificate-Service-Communications-Primary-AboutToExpire    NotRun
    Test-Certificate-Token-Decrypting-Primary-NotFoundInStore        NotRun
    Test-Certificate-Token-Decrypting-Primary-IsSelfSigned           NotRun
    Test-Certificate-Token-Decrypting-Primary-PrivateKeyAbsent       NotRun
    Test-Certificate-Token-Decrypting-Primary-Expired                NotRun
    Test-Certificate-Token-Decrypting-Primary-Revoked                NotRun
    Test-Certificate-Token-Decrypting-Primary-AboutToExpire          NotRun
    Test-Certificate-Token-Signing-Primary-NotFoundInStore           NotRun
    Test-Certificate-Token-Signing-Primary-IsSelfSigned              NotRun
    Test-Certificate-Token-Signing-Primary-PrivateKeyAbsent          NotRun
    Test-Certificate-Token-Signing-Primary-Expired                   NotRun
    Test-Certificate-Token-Signing-Primary-Revoked                   NotRun
    Test-Certificate-Token-Signing-Primary-AboutToExpire             NotRun
    Test-Certificate-SSL-Primary-NotFoundInStore                     NotRun
    Test-Certificate-SSL-Primary-IsSelfSigned                        NotRun
    Test-Certificate-SSL-Primary-PrivateKeyAbsent                    NotRun
    Test-Certificate-SSL-Primary-Expired                             NotRun
    Test-Certificate-SSL-Primary-Revoked                             NotRun
    Test-Certificate-SSL-Primary-AboutToExpire                       NotRun
    Test-Certificate-Token-Decrypting-Secondary-NotFoundInStore      NotRun
    Test-Certificate-Token-Decrypting-Secondary-IsSelfSigned         NotRun
    Test-Certificate-Token-Decrypting-Secondary-PrivateKeyAbsent     NotRun
    Test-Certificate-Token-Decrypting-Secondary-Expired              NotRun
    Test-Certificate-Token-Decrypting-Secondary-Revoked              NotRun
    Test-Certificate-Token-Decrypting-Secondary-AboutToExpire        NotRun
    Test-Certificate-Token-Signing-Secondary-NotFoundInStore         NotRun
    Test-Certificate-Token-Signing-Secondary-IsSelfSigned            NotRun
    Test-Certificate-Token-Signing-Secondary-PrivateKeyAbsent        NotRun
    Test-Certificate-Token-Signing-Secondary-Expired                 NotRun
    Test-Certificate-Token-Signing-Secondary-Revoked                 NotRun
    Test-Certificate-Token-Signing-Secondary-AboutToExpire           NotRun
    CheckFarmDNSHostResolution                                       NotRun
    CheckDuplicateSPN                                                NotRun
    TestServiceAccountProperties                                       Pass
    TestAppPoolIDMatchesServiceID                                    NotRun
    TestComputerNameEqFarmName                                       NotRun
    TestSSLUsingADFSPort                                             NotRun
    TestSSLCertSubjectContainsADFSFarmName                           NotRun
    TestAdfsAuditPolicyEnabled                                         Pass
    TestAdfsRequestToken                                               Pass
    CheckOffice365Endpoints                                          NotRun
    TestADFSO365RelyingParty                                         NotRun
    TestNtlmOnlySupportedClientAtProxyEnabled                        NotRun

    I appreciate any help.

       
    Monday, December 11, 2017 10:08 AM