none
Probably a MaxtokenSize problem but...

    Question

  • Hi,

    I’m working for an authority with +2000 users, 12 2003 DC´s and the Domain and Forest functional levels are Windows Server 2003 (MaxTokenSize 12000). It’s an old AD and many users are members of way to many groups. Now some users have problems with Kerberos authentication. When they log on to their Windows XP computers event 1053 with source “Userenv” occurs, group-policies not applies and they are not able to search in AD without getting a message about “not enough available space…”.

    This sounds like a problem with MaxTokenSize or Tokenbloat but I can’t see why this should occur. Here is an example of a user with the problem:

    User A is member of 404 (many nestled) groups, have no SidHistory and the account is not trusted for delegation and the tokensize is 8745 bytes. I have used TokenSZ to get this value but I’ve also verified it with the “1200 + 40d + 8s” formula.

    8745 bytes is far from 12000 bytes and 404 SID’s is far from 1024 SID’s.

    I remove 4 groups from the user and the total amount of groups is now 350 and the Tokensize is 8249 bytes. Now is everything working like it shall!

    I have not tried to raise the MaxTokenSize because it’s not an easy task to accomplish political in this organization. However if I could prove that we crossed the limits (12000 or 1024) I could request for a change but not now. Can someone explain to me why we are getting these problems when we not exceeding the values?

    Best regards
    Mattias Johansson  


    • Edited by --M-- Wednesday, August 24, 2011 11:58 AM
    Wednesday, August 24, 2011 10:26 AM

Answers

  • See this

    http://support.microsoft.com/kb/327825


    Best regards Biswajit Biswas Disclaimer: This posting is provided "AS IS" with no warranties or guarantees , and confers no rights. MCP 2003,MCSA 2003, MCSA:M 2003, CCNA, MCTS, Enterprise Admin
    Wednesday, August 24, 2011 10:46 AM
  • Hi,

    I was going through couple of links to explore more on your issue and found this http://blogs.technet.com/b/shanecothran/archive/2010/07/16/maxtokensize-and-kerberos-token-bloat.aspx

    The link which is provided by Martin has a documentation on " how this is happening"

    Since you already found you're having issues while crossing 400 Groups.. Did you try the hotfix avalbl(mentioned by biswajit),, that you can apply only on DC's.

    However it is worth re-consolidating the groups in your environment..

    Similiar issue hapedn in my enviornment once, where in there were t00 many security groups in domain which are no longer needed access or the proj team is moved, in these cases, we started sending emails to users who raised a request to create these  security groups. Of course,it is a long process but it is worth spending time here..

     


    Regards, Mohan R Sr. Administrator - Server Support
    Wednesday, August 24, 2011 11:12 AM
  • Have you taken into account group nesting which will increase the group count.  If you want to expand the Token limit I would suggest using Group Policy.

    (44 bytes x 400 groups) + 500 bytes = 18100 which is greater than 12,000
    http://support.microsoft.com/kb/912376

     

    How to calculate token size

    Access token size in bytes can be estimated by using the following formula:

    [12 x number of user rights] + [token overhead] + [44 x number of group memberships] = token size in bytes

     

    • User rights include rights such as "Log on locally" or "Access this computer from the network." The only user rights that are added to an access token are those user rights that are configured on the server that hosts a secured resource. Most Exchange Server users are likely to have only two or three user rights on the Exchange server. Administrators may have dozens of user rights. Each user right requires 12 bytes to store it in the token.
    • Token overhead includes multiple fields such as the token source, expiration time, and impersonation information. For a typical domain user who has no special access or restrictions, token overhead is likely to be between 400 and 500 bytes. Typically, estimating 500 bytes for both user rights and token overhead is more than sufficient.
    • Each group membership adds the group SID to the token together with an additional 16 bytes for associated attributes and information. The maximum possible size for a SID is 68 bytes. However, it is rare for a SID to be this large. In Windows Server 2003 and in earlier versions of Windows, the typical SID for a user or a group is 28 bytes in length. Therefore, each security group to which a user belongs typically adds 44 bytes to the user's token size.

     

    --
    Paul Bergson
    MVP - Directory Services
    MCITP: Enterprise Administrator
    MCTS, MCT, MCSE, MCSA, Security+, BS CSci
    2008, Vista, 2003, 2000 (Early Achiever), NT4
    http://www.pbbergs.com    Twitter @pbbergs
    http://blogs.dirteam.com/blogs/paulbergson

    Please no e-mails, any questions should be posted in the NewsGroup. This posting is provided "AS IS" with no warranties, and confers no rights.

     


    Wednesday, August 24, 2011 12:16 PM
    Moderator
  • Here is one possible explanation (as per http://blogs.technet.com/b/surama/archive/2009/04/06/kerberos-authentication-problem-with-active-directory.aspx )

    "If a token is less than 4 KB, the amount of kernel memory that is allocated for it is exactly what is required to hold the token. (...)

    But if a token is even slightly larger than 4 KB (4096 bytes) the amount of memory that is allocated per copy will jump to exactly 8 KB (8192 bytes). If a token is even slightly larger than 8 KB, the memory allocation will jump to exactly 12 KB. Therefore every time the token sizes crosses one of these critical 4-KB boundaries, there is a sudden jump in the use of paged pool memory and user will have intermitted results."

    This might not correspond EXACTLY to the values you are seeing, but I'm not sure to what extent these values represent the actual token size. More importantly, it explains why a minor change in the 8k range resolves the issue...

    hth
    Marcin

     



    Wednesday, August 24, 2011 12:25 PM

All replies

  • See this

    http://support.microsoft.com/kb/327825


    Best regards Biswajit Biswas Disclaimer: This posting is provided "AS IS" with no warranties or guarantees , and confers no rights. MCP 2003,MCSA 2003, MCSA:M 2003, CCNA, MCTS, Enterprise Admin
    Wednesday, August 24, 2011 10:46 AM
  • You can view some important information about such problems in the "Addressing Problems Due to Access Token Limitation Document"

    (http://www.microsoft.com/download/en/details.aspx?DisplayLang=en&id=13749)

     

    Wednesday, August 24, 2011 10:53 AM
  • Can you clarify how you came up with these numbers?

    For starters, 12000kb is roughly 12MB - and that's by far larger than any value I've seen (looks like you have kB and B mixed up).

    In addition, with 400 groups, the formula will yield the value that's larger than 12kB

    hth
    Marcin

     


    Wednesday, August 24, 2011 11:08 AM
  • Hi,

    I was going through couple of links to explore more on your issue and found this http://blogs.technet.com/b/shanecothran/archive/2010/07/16/maxtokensize-and-kerberos-token-bloat.aspx

    The link which is provided by Martin has a documentation on " how this is happening"

    Since you already found you're having issues while crossing 400 Groups.. Did you try the hotfix avalbl(mentioned by biswajit),, that you can apply only on DC's.

    However it is worth re-consolidating the groups in your environment..

    Similiar issue hapedn in my enviornment once, where in there were t00 many security groups in domain which are no longer needed access or the proj team is moved, in these cases, we started sending emails to users who raised a request to create these  security groups. Of course,it is a long process but it is worth spending time here..

     


    Regards, Mohan R Sr. Administrator - Server Support
    Wednesday, August 24, 2011 11:12 AM
  • Of course I meant byte...
    The values are generated with this command "tokensz /compute_tokensize /dump_groups".
    I don’t remember the exact values but something like “1200+(40x130DLG)+(8x280GG)=1200+5200+2240=8640


    Wednesday, August 24, 2011 11:55 AM
  • Thanks for the links but I've seen all of them before and I can’t find anything explaining my problems in them. I will read them all again and see If I missed anything. If you did find anything similar to my scenario in them please let me know exactly what.

    About the hotfix: “Note Windows 7, Windows Server 2008 R2, Windows Server 2003, Windows Vista, Windows Server 2008, and Windows XP Professional include a fix for this problem.”

    Wednesday, August 24, 2011 12:08 PM
  • Have you taken into account group nesting which will increase the group count.  If you want to expand the Token limit I would suggest using Group Policy.

    (44 bytes x 400 groups) + 500 bytes = 18100 which is greater than 12,000
    http://support.microsoft.com/kb/912376

     

    How to calculate token size

    Access token size in bytes can be estimated by using the following formula:

    [12 x number of user rights] + [token overhead] + [44 x number of group memberships] = token size in bytes

     

    • User rights include rights such as "Log on locally" or "Access this computer from the network." The only user rights that are added to an access token are those user rights that are configured on the server that hosts a secured resource. Most Exchange Server users are likely to have only two or three user rights on the Exchange server. Administrators may have dozens of user rights. Each user right requires 12 bytes to store it in the token.
    • Token overhead includes multiple fields such as the token source, expiration time, and impersonation information. For a typical domain user who has no special access or restrictions, token overhead is likely to be between 400 and 500 bytes. Typically, estimating 500 bytes for both user rights and token overhead is more than sufficient.
    • Each group membership adds the group SID to the token together with an additional 16 bytes for associated attributes and information. The maximum possible size for a SID is 68 bytes. However, it is rare for a SID to be this large. In Windows Server 2003 and in earlier versions of Windows, the typical SID for a user or a group is 28 bytes in length. Therefore, each security group to which a user belongs typically adds 44 bytes to the user's token size.

     

    --
    Paul Bergson
    MVP - Directory Services
    MCITP: Enterprise Administrator
    MCTS, MCT, MCSE, MCSA, Security+, BS CSci
    2008, Vista, 2003, 2000 (Early Achiever), NT4
    http://www.pbbergs.com    Twitter @pbbergs
    http://blogs.dirteam.com/blogs/paulbergson

    Please no e-mails, any questions should be posted in the NewsGroup. This posting is provided "AS IS" with no warranties, and confers no rights.

     


    Wednesday, August 24, 2011 12:16 PM
    Moderator
  • Here is one possible explanation (as per http://blogs.technet.com/b/surama/archive/2009/04/06/kerberos-authentication-problem-with-active-directory.aspx )

    "If a token is less than 4 KB, the amount of kernel memory that is allocated for it is exactly what is required to hold the token. (...)

    But if a token is even slightly larger than 4 KB (4096 bytes) the amount of memory that is allocated per copy will jump to exactly 8 KB (8192 bytes). If a token is even slightly larger than 8 KB, the memory allocation will jump to exactly 12 KB. Therefore every time the token sizes crosses one of these critical 4-KB boundaries, there is a sudden jump in the use of paged pool memory and user will have intermitted results."

    This might not correspond EXACTLY to the values you are seeing, but I'm not sure to what extent these values represent the actual token size. More importantly, it explains why a minor change in the 8k range resolves the issue...

    hth
    Marcin

     



    Wednesday, August 24, 2011 12:25 PM
  • Yes the number 404 is including the nested groups. The number is from TokenSZ which lists SID’s for all groups.

    I haven’t seen the formula you show here earlier. I’ve used this formula:

    TokenSize = 1200 + 40d + 8s

    This formula uses the following values:

    ·         d: The number of domain local groups a user is a member of plus the number of universal groups outside the user's account domain plus the number of groups represented in security ID (SID) history.

    ·         s: The number of security global groups that a user is a member of plus the number of universal groups in a user's account domain.

    ·         1200: The estimated value for ticket overhead. This value can vary depending on factors such as DNS domain name length, client name, and other factors.

    This one is from http://support.microsoft.com/kb/327825

    If I understand this right you only need 8 bytes for Global Groups and in my scenario 280 of the groups are global. And if I use the formula you provided at the scenario when the problem not occur it will look like this:

    (12x19 user rights)+(500 overhead)+(44x350 groups)=16128

    This also far over 12000 but there is no problems!

    However I’m not sure the different ways to calculate this matter as long as TokenSZ reports 8745 bytes? I mean this tool is provided by Microsoft to calculate these things so I hope it gives correct values or am I missing something?
    Wednesday, August 24, 2011 12:46 PM
  • I've seen SIDs of 24, 16, and 12 bytes, but most are 28. Most users are members of builtin groups, which have 16 byte SIDs. I wonder how your results from TokenSZ would compare to the size of the tokenGroups attribute of the user. The following PowerShell script retrieves the number of security groups in the collection (which includes all security group memberships, including due to nesting, plus the primary group) and the total number of bytes. If there is an overhead of 16 bytes per group, you could take the total bytes reported and add 16 times the number of groups.

     

    # Bind to the user object in Active Directory.
    $User = [ADSI]"LDAP://cn=Jim Smith,ou=West,dc=MyDomain,dc=com"

    # Retrieve tokenGroups attribute, which is operational (constructed).
    $User.psbase.RefreshCache("tokenGroups")
    $SIDs = $User.Properties.Item("tokenGroups")
    # Enumerate security groups. Add up SID bytes and number of groups.
    $Count = 0
    $Bytes = 0
    ForEach ($Value In $SIDs)
    {
        $SID = New-Object System.Security.Principal.SecurityIdentifier $Value, 0
        $Bytes = $Bytes + $SID.BinaryLength
        $Count = $Count + 1
    }

    "Number of Groups: $Count"
    "Number of Bytes: $Bytes"

    -----

     

    Seems to me the tokenGroups attribute should correspond to the security token.

     


    Richard Mueller - MVP Directory Services
    Wednesday, August 24, 2011 4:40 PM
  • Marcin Polich: This is news to me and I've never seen it before. As you mention this can explain the behavior but I still think TokenSZ reports the actual value. So in my case 8745 will be handled as 12000 and the limit is over 12000 so it should not be a problem. It would be really nice if Microsoft could provide detailed information about this!

     

    Richard Mueller:  I will visit this customer next time at Monday and will then try your script and report back here.

    Friday, August 26, 2011 8:56 AM
  • If your limit is set to 12000, then the next threshold would be at 12kB - that translates into 12288 - which would explain the behavior you are seeing (i.e. increasing group membership around 8kB threshold would lead to token issues).

    Btw. the article I referenced is published on Technet - which I'd consider to be a Microsoft-based source...

    hth
    Marcin

    Friday, August 26, 2011 10:55 AM