locked
Intermittent Issue RRS feed

  • Question

  • Experiencing an intermittent issue since yesterday whereby internal users who would normally be redirected straight through to o365 with IE are getting a username/password pop up box. When they fill this in the password is seemingly rejected however this isn't recorded in any of the logs that I can see. If the user uses Edge or Chrome which uses form based authentication it logs in ok. On some computers the login works fine a bit confused where to start with this.

    Overnight we have upgraded from 2012 R2 to 2016 for ADFS which hasn't helped

    Robbie

    Friday, October 19, 2018 12:02 PM

Answers

  • Channel Binding should take place only for domain joined computer using supported browser used from the internal network (not going through a WAP server).

    The problem is your webfilter. It breaks the TLS tunnel and rebuild it. As a result, you break the channel binding token. Channel Binding is a taking advantage of the TLS encryption keys (sort of) to secure the Windows Integrated Authentication traffic. If you breaks the tunnel somewhere in the middle, both parties are using different keys which is basically seen by the browser as a man in the middle attack and breaks the auth.

    There is little or no value of doing such filtering for ADFS traffic. You should just disable it and make sure there is no TLS inspection or anything of the sort between the client and the ADFS servers.


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    • Marked as answer by GPP Tech Saturday, October 27, 2018 4:30 PM
    Sunday, October 21, 2018 5:43 PM
  • I have figured this out ESET Endpoint AV was intercepting HTTPS i.e. 'Man in the middle'. I have added an exception rule and now all is working ok thanks for the help

    • Marked as answer by GPP Tech Saturday, October 27, 2018 4:29 PM
    Saturday, October 27, 2018 4:29 PM

All replies

  • Can you share the logs when a user fails to authenticate?


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Friday, October 19, 2018 12:32 PM
  • I've been unable to locate a corresponding log entry under security or the adfs section
    Friday, October 19, 2018 12:33 PM
  • Oh wait, you are getting these pop-up in Outlook?

    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Friday, October 19, 2018 12:41 PM
  • In Internet Explorer


    • Edited by GPP Tech Friday, October 19, 2018 12:51 PM
    Friday, October 19, 2018 12:48 PM
  • What was odd was the user tried 3 computers and they all did the same. There are no roaming profiles to follow them around. They did it in our office and it worked but back to other PC's and it didn't. Our test account worked fine in both locations
    Friday, October 19, 2018 12:52 PM
  • Well, this pop-up is the Windows Integrated Authentication pop-up.

    Only on-premises and domain joined computer are supposed to do Windows Integrated Authentication. And the browser will have to be enabled for Windows Integrated Authentication too.

    Make sure that:

    1. If the clients are connected externally, that the URL of your ADFS servers when resolved from an Internet endpoint points to the public IP address of your WAP servers (not the ADFS servers).

    2. If the clients are on-prem and domain-joined, make sure the URL of the ADFS is listed in the Intranet Trusted Zone (or a zone where Windows Integrated Authentication is enabled).


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Friday, October 19, 2018 12:59 PM
  • I've managed to replicate it on the same PC the user was on.

    It is internal and domain joined.

    Site is showing as local intranet 

    Time matches the ADFS server to the second

    No logs in ADFS that correspond to the request. Pressing ok at login results in prompting for password again

    Robbie

    Friday, October 19, 2018 1:03 PM
  • Just found this error on the PC in question however I think it is a red herring as occurred after the issues seen

    Faulting application name: IEXPLORE.EXE, version: 11.0.17134.1, time stamp: 0x6639744d
    Faulting module name: Windows.UI.XamlHost.dll, version: 10.0.17134.319, time stamp: 0x8341feb9
    Exception code: 0xc0000409
    Fault offset: 0x000058cf
    Faulting process ID: 0x73c
    Faulting application start time: 0x01d467ab07e9bdcf
    Faulting application path: C:\Program Files (x86)\Internet Explorer\IEXPLORE.EXE
    Faulting module path: C:\Windows\System32\Windows.UI.XamlHost.dll
    Report ID: 969732e8-5e31-4d13-9294-064b874b85ae
    Faulting package full name:


    • Edited by GPP Tech Friday, October 19, 2018 1:36 PM
    Friday, October 19, 2018 1:31 PM
  • Checked password lockout and no bad passwords reaching AD
    Friday, October 19, 2018 1:42 PM
  • Enabled tracing which is only showing me the requests reaching ADFS not much help still

    Log Name:      Security
    Source:        AD FS Auditing
    Date:          19/10/2018 15:27:03
    Event ID:      403
    Task Category: (3)
    Level:         Information
    Keywords:      Classic,Audit Success
    User:          EMPIRE\adfs
    Computer:      ADFS16.xxx.xxxx
    Description:
    An HTTP request was received. See audit 510 with the same Instance ID for headers.
    Instance ID: 894941d1-2156-45e5-be91-11af6c80dd1e
    Activity ID: 67bcb9bc-b20b-434d-a38f-74ec7e3653b5
    Request Details:
        Date And Time: 2018-10-19 14:27:03
        Client IP: 172.16.0.33
        HTTP Method: GET
        Url Absolute Path: /adfs/ls/wia
        Query string: ?mkt=en-GB&client-request-id=67bcb9bc-b20b-434d-a38f-74ec7e3653b5&username=xx%40xxxx.xxx.uk&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx=LoginOptions%3D3%26estsredirect%3d2%26estsrequest%3drQIIAXWSsW_TUBDG46QNbUEQIRCIqQMTyI79nu3UkSrRJnGaENtN4yS1l8h2nPglsV9qv2DwwMxGN6SOgITUsRISoC5l7NS5IxNCDKgTI-xxxx39Z1_lDlfLtYbSB9L2zAaLXjf6hvOolOHQGrtlpSnYxONS3gH7pKWZJPGJixzh3nqMJ__k6fe3Mh8W_0fA5dr91J2NuiUJk5a50BZEMssMP8B0&cbcxt=&lc=
        Local Port: 443
        Local IP: 172.16.222.12
        User Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)
        Content Length: 0
        Caller Identity: -
        Certificate Identity (if any): -
        Targeted relying party: -
        Through proxy: False
        Proxy DNS name: -
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="AD FS Auditing" />
        <EventID Qualifiers="0">403</EventID>
        <Level>0</Level>
        <Task>3</Task>
        <Keywords>0x80a0000000000000</Keywords>
        <TimeCreated SystemTime="2018-10-19T14:27:03.031788300Z" />
        <EventRecordID>17848</EventRecordID>
        <Channel>Security</Channel>
        <Computer>ADFS16.xxx</Computer>
        <Security UserID="S-1-5-21-220523388-1547161642-725345543-95684" />
      </System>
      <EventData>
        <Data>894941d1-2156-45e5-be91-11af6c80dd1e</Data>
        <Data>67bcb9bc-b20b-434d-a38f-74ec7e3653b5</Data>
        <Data>2018-10-19 14:27:03</Data>
        <Data>172.16.0.33</Data>
        <Data>GET</Data>
        <Data>/adfs/ls/wia</Data>
        <Data>?mkt=en-GB&amp;client-request-id=67bcb9bc-b20b-434d-a38f-74ec7e3653b5&amp;username=xxxx%40xxxx.xx.xx&amp;wa=wsignin1.0&amp;wtrealm=urn%3afederation%3aMicrosoftOnline&amp;wctx=LoginOptions%3D3%26estsredirect%3d2%26estsrequest%3drQIIAXWSsW_TUBDG46QNbUEQIRCIqQMTyI79nu3UkSrRJnGaENtN4yS1l8h2nPglsV9qv2DwwMxGN6SOgITUsRISoC5l7NS5IxNCDKgTI-xxxxxxxxx&amp;cbcxt=&amp;lc=</Data>
        <Data>443</Data>
        <Data>172.16.222.12</Data>
        <Data>Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)</Data>
        <Data>0</Data>
        <Data>-</Data>
        <Data>-</Data>
        <Data>-</Data>
        <Data>False</Data>
        <Data>-</Data>
      </EventData>
    </Event>

    Friday, October 19, 2018 2:39 PM
  • Well, that is no longer an ADFS issue then :)

    Do other websites with integrated authentication have the same issue?


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Friday, October 19, 2018 3:47 PM
  • I'm not so quick to rule it out there are no other services that use integrated authentication effected. I have an o365 support case open. Will feedback any results if of course this thread doesn't solve it sooner. Note there are no failure or success audits for the user login

    Robbie

    Saturday, October 20, 2018 8:01 AM
  • This event may be associated as on the exact same time stamp as a request but could it be linked to a login prior? Any clues here?

    Log Name:      Security
    Source:        Microsoft-Windows-Security-Auditing
    Date:          20/10/2018 11:16:19
    Event ID:      4625
    Task Category: Logon
    Level:         Information
    Keywords:      Audit Failure
    User:          N/A
    Computer:      ADFS16xxxxx
    Description:
    An account failed to log on.
    Subject:
     Security ID:  NULL SID
     Account Name:  -
     Account Domain:  -
     Logon ID:  0x0
    Logon Type:   3
    Account For Which Logon Failed:
     Security ID:  NULL SID
     Account Name:  
     Account Domain:  
    Failure Information:
     Failure Reason:  An Error occured during Logon.
     Status:   0xC000035B
     Sub Status:  0x0
    Process Information:
     Caller Process ID: 0x0
     Caller Process Name: -
    Network Information:
     Workstation Name: -
     Source Network Address: -
     Source Port:  -
    Detailed Authentication Information:
     Logon Process:  Kerberos
     Authentication Package: Kerberos
     Transited Services: -
     Package Name (NTLM only): -
     Key Length:  0
    This event is generated when a logon request fails. It is generated on the computer where access was attempted.
    The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
    The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).
    The Process Information fields indicate which account and process on the system requested the logon.
    The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
    The authentication information fields provide detailed information about this specific logon request.
     - Transited services indicate which intermediate services have participated in this logon request.
     - Package name indicates which sub-protocol was used among the NTLM protocols.
     - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
        <EventID>4625</EventID>
        <Version>0</Version>
        <Level>0</Level>
        <Task>12544</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8010000000000000</Keywords>
        <TimeCreated SystemTime="2018-10-20T10:16:19.623903900Z" />
        <EventRecordID>337238</EventRecordID>
        <Correlation ActivityID="{71D475FF-685A-0003-BA76-D4715A68D401}" />
        <Execution ProcessID="632" ThreadID="1916" />
        <Channel>Security</Channel>
        <Computer>ADFS16xxx</Computer>
        <Security />
      </System>
      <EventData>
        <Data Name="SubjectUserSid">S-1-0-0</Data>
        <Data Name="SubjectUserName">-</Data>
        <Data Name="SubjectDomainName">-</Data>
        <Data Name="SubjectLogonId">0x0</Data>
        <Data Name="TargetUserSid">S-1-0-0</Data>
        <Data Name="TargetUserName">
        </Data>
        <Data Name="TargetDomainName">
        </Data>
        <Data Name="Status">0xc000035b</Data>
        <Data Name="FailureReason">%%2304</Data>
        <Data Name="SubStatus">0x0</Data>
        <Data Name="LogonType">3</Data>
        <Data Name="LogonProcessName">Kerberos</Data>
        <Data Name="AuthenticationPackageName">Kerberos</Data>
        <Data Name="WorkstationName">-</Data>
        <Data Name="TransmittedServices">-</Data>
        <Data Name="LmPackageName">-</Data>
        <Data Name="KeyLength">0</Data>
        <Data Name="ProcessId">0x0</Data>
        <Data Name="ProcessName">-</Data>
        <Data Name="IpAddress">-</Data>
        <Data Name="IpPort">-</Data>
      </EventData>
    </Event>

    Saturday, October 20, 2018 10:22 AM
  • Made a step forward the issue appears to be with channel binding. This setting "fixes" the issue but doesn't get to the root cause.

    set-adfsproperties extendedprotectiontokencheck none

    When the call to ADFS is happening can someone explain what happens? We have a web filter with decrypt and inspect going out to the internet but there "shouldnt be" any man in the middle on the local network.

    Is ADFS going out to the internet during this process? or is there another explanation such as a Kerberos fault on the network?

    For others that may experience the issue this doc helped me

    https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/troubleshooting/ad-fs-tshoot-iwa

    Robbie



    • Edited by GPP Tech Sunday, October 21, 2018 9:41 AM
    Sunday, October 21, 2018 9:34 AM
  • Channel Binding should take place only for domain joined computer using supported browser used from the internal network (not going through a WAP server).

    The problem is your webfilter. It breaks the TLS tunnel and rebuild it. As a result, you break the channel binding token. Channel Binding is a taking advantage of the TLS encryption keys (sort of) to secure the Windows Integrated Authentication traffic. If you breaks the tunnel somewhere in the middle, both parties are using different keys which is basically seen by the browser as a man in the middle attack and breaks the auth.

    There is little or no value of doing such filtering for ADFS traffic. You should just disable it and make sure there is no TLS inspection or anything of the sort between the client and the ADFS servers.


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    • Marked as answer by GPP Tech Saturday, October 27, 2018 4:30 PM
    Sunday, October 21, 2018 5:43 PM
  • Channel Binding should take place only for domain joined computer using supported browser used from the internal network (not going through a WAP server).

    The problem is your webfilter. It breaks the TLS tunnel and rebuild it. As a result, you break the channel binding token. Channel Binding is a taking advantage of the TLS encryption keys (sort of) to secure the Windows Integrated Authentication traffic. If you breaks the tunnel somewhere in the middle, both parties are using different keys which is basically seen by the browser as a man in the middle attack and breaks the auth.

    There is little or no value of doing such filtering for ADFS traffic. You should just disable it and make sure there is no TLS inspection or anything of the sort between the client and the ADFS servers.


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    I think you misunderstand, the problematic computers are domain joined and on the internal network. The decrypt and inspect on the web filter is on the edge and only happens to when traffic goes to the web not internal. WAP is not called internally.

    After reading more it seems the issue is occurring between the computer and the ADFS server which seems impossible as it is a direct route with no proxies involved so I am a little confused.

    We are seeing an issue on our web filter that is supposed to ID all users by Kerberos whereby some users aren't being authenticated and wondered if there is a link. Will check if that filter does an extended check too

    Robbie

    Monday, October 22, 2018 6:42 AM
  • I am confused indeed.

    I don't understand anymore :(

    Which situations are you seeing this behavior?

    1. Domain-joined machine using ADFS

    2. Domain-joined machine using WAP

    Channel Binding should take place only on 1. As external clients (scenario 2) are not using WIA.

    For 1, you need to ensure than the browser supports the feature and that you don't have a device between the client and the ADFS servers (sometimes, HLB are doing TLS termination).


    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Monday, October 22, 2018 3:30 PM
  • 1) Domain joined machine using ADFS

    There is nothing between the machine and ADFS it should be a direct route. Clients are Windows 10 with the latest updates

    Robbie

    Monday, October 22, 2018 3:45 PM
  • I have figured this out ESET Endpoint AV was intercepting HTTPS i.e. 'Man in the middle'. I have added an exception rule and now all is working ok thanks for the help

    • Marked as answer by GPP Tech Saturday, October 27, 2018 4:29 PM
    Saturday, October 27, 2018 4:29 PM