locked
NPS on Windows Server 2008 R2: Reason code 266 solved with kb article for Windows Server 2003 - but why? RRS feed

  • Question

  • Hi everyone,

    a customer of mine wants to deploy 802.1x wired authentication in 70+ locations. So I set up a test lab and started playing. Eventually, I had my Cisco Catalyst Switch 3560 (12.2(55) IP-Base image) and my NPS server on Windows Server 2008 R2 up and running. The test client got certificates and all ... But it did not authenticate. Instead, I got reason code 266 "The message received was unexpected or badly formatted." 

    So I googled a bit and found this old kb article http://support.microsoft.com/kb/933430/en-us. In the workarounds section I used method 3 on my NPS, which modifies the behavior of the SCHANNEL provider. This was indicated by another post on this forum (sorry, lost the link). Surprisingly, it worked! - Now I wonder why?

    Does this registry setting effect the security of the TLS session in a negative way? I do not want to roll out this "fix", unless I have a clear understanding of the security implications.

    Any feedback is welcome!


    ----------------------- Greetings from Germany, Martin

    Tuesday, February 3, 2015 10:35 AM

Answers

  • Hi Martin,

    >>Surprisingly, it worked! - Now I wonder why?

    This issue is caused by the size limitation of the list of trusted certificate authorities.

    If the size of this list exceeds 12,228 bytes, Schannel logs Warning event ID 36885. Then, Schannel truncates the list of trusted root certificates and sends this truncated list to the client computer. Then issue occurs.

    When we change the registry in method 3, the server will not send this list anymore. The client will display all of the certificate installed.

    >>Does this registry setting effect the security of the TLS session in a negative way?

    This behavior may affect how the client responds to a request for a certificate.

    For example, if Internet Explorer receives a request for client authentication, Internet Explorer displays only the client certificates that appear in the chain of one of the certification  authorities that are in the list from the server. However, if the server does not send a list of trusted certificate authorities, Internet Explorer displays all the client certificates that are installed on the client computer.

    Besides, the article has said that the workarounds described below will apply to Windows Server 2008 and Windows Server 2008 R2 as well.

    Best Regards.


    Steven Lee Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact tnmff@microsoft.com.

    Wednesday, February 4, 2015 9:51 AM

All replies

  • Hi Martin,

    >>Surprisingly, it worked! - Now I wonder why?

    This issue is caused by the size limitation of the list of trusted certificate authorities.

    If the size of this list exceeds 12,228 bytes, Schannel logs Warning event ID 36885. Then, Schannel truncates the list of trusted root certificates and sends this truncated list to the client computer. Then issue occurs.

    When we change the registry in method 3, the server will not send this list anymore. The client will display all of the certificate installed.

    >>Does this registry setting effect the security of the TLS session in a negative way?

    This behavior may affect how the client responds to a request for a certificate.

    For example, if Internet Explorer receives a request for client authentication, Internet Explorer displays only the client certificates that appear in the chain of one of the certification  authorities that are in the list from the server. However, if the server does not send a list of trusted certificate authorities, Internet Explorer displays all the client certificates that are installed on the client computer.

    Besides, the article has said that the workarounds described below will apply to Windows Server 2008 and Windows Server 2008 R2 as well.

    Best Regards.


    Steven Lee Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact tnmff@microsoft.com.

    Wednesday, February 4, 2015 9:51 AM
  • Hi Martin,

    We've not heard from you yet. I assume the information provided by me has helped. I am marking my reply as answered now.

    In case the information did not help, please feel free to unmark the answer and come back to us with your comments.

    Best Regards.


    Steven Lee Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact tnmff@microsoft.com.

    Thursday, March 5, 2015 5:47 AM
  • Hi Steven,

    thanks for Information. Indeed it did answer my question. Sorry, that I did not post an immediate reply. I'll promise I'll do it next time right away.


    ----------------------- Greetings from Germany, Martin

    Friday, March 6, 2015 7:55 AM