none
Retrieve nested LDAP groups independent from the network env. (five different approaches) RRS feed

  • Question

  • Hi all,

    I want to retrieve a list of nested LDAP groups per user from the Active Directory. I have been searching google for half a day now, but I'm still not sure what approach to use. I have the following requirements:

    * The script/program must run in different network environments (I can't be sure if there is a global catelog or AD DS or AD LDS, etc). I will write my own program.
    * The membership info will be used in combination with directory ACL's and must be as complete as possible (global groups, universal groups, local groups, perhaps different domains). Distribution groups are not really necessary, because they are not used in the directory ACL's.
    * It would be nice to support other LDAP implementations than Active Directory using the same code, but that not a hard requirement. I could use another approach to support a different LDAP.

    Now I have figured out five possible approaches (info comes from different sites, please correct me if I'm wrong):

    1) tokengroups attribute:
    - The attribute contains Univeral groups of the forest, global groups from the local domain, domain local groups from the local domain (assuming native mode) and local groups from the local machine.
    - Returns a list of SIDs which will have to be translated to group names
    - The tokenGroups attribute exists on both AD DS and AD LDS
    - For AD DS, the tokenGroups attribute is not present if no GC server is available to evaluate the transitive reverse memberships.
    - quote from site "Now that I have had a chance to test it though I can definitely say that tokenGroups WILL get the Universal groups from the other domains even if is NOT a GC. I just did it in my test lab."
    - Token Groups cannot be retrieved if no Global Catalog is present to retrieve the transitive reverse memberships.

    2) tokenGroupsGlobalAndUniversal
    - A subset of the tokenGroups attribute. Only the global and universal group SIDs are included.
    - If you want consistent results, read tokenGroupsGlobalAndUniversal that will return the same result no matter which DC you are connected to. However, it will not include local groups.
    - other source says "tokenGroups will give you all the security groups this user belongs to, including nested groups and domain users, users, etc tokenGroupsGlobalAndUniversal will include everything from tokenGroups AND distribution groups". Not sure if this is correct, I think it doesn't contain local groups.
    - The tokenGroupsGlobalAndUniversal attribute exists on AD DS but not on AD LDS.

    3) LDAP_MATCHING_RULE_IN_CHAIN / 1.2.840.113556.1.4.1941
    - Use a recursive search query which returns all nested groups for user at once.
    - Returns all groups except for the primary group
    - It's a fast approach, see performance test from Richard Mueller:
    http://social.technet.microsoft.com/Forums/fr-FR/f238d2b0-a1d7-48e8-8a60-542e7ccfa2e8/recursive-retrieval-of-all-ad-group-memberships-of-a-user?forum=ITCG
    - It only works on Active Directory, not for other LDAP implementations

    4) Recursive retrieval of the memberOf attribute
    - Retrieves all groups except the primary group. (also local groups from other domains??)
    - works for all LDAP implementations
    - executes a lot of queries to the LDAP, especially if you want to scan all users/groups (perhaps limited on OU, but still)

    5) Store memberOf attribute in local database and calculate the nested groups using recursive queries to the local database
    - No heavy load to the LDAP
    - Needs space to store the user/group info locally (embedded Derby database perhaps)
    - Performs fast since the queries are executed locally
    - Works for all LDAP implementations

    My thoughts on these different approaches:
    * appreach 1) I understand that the tokengroups attribute is not present if no GC server is available. In how many network environments is this the case? This option won't work because I want to support different network environments.
    * approach 2) The tokenGroupsGlobalAndUniversal attribute exists on AD DS but not on AD LDS. Same here, in how many network environments is this the case? I don't think I can rely on this approach.
    * approach 3) Seems to be a good option. How will it perform compared to approach 5 (local recursive queries)? Won't work for other LDAP implementations
    * approach 4) I don't think I want to execute that many queries to the LDAP. I can limit the scan on OU, but still companies can have thousands of users and groups.
    * approach 5) Perhaps the best approach. I want to store user/group info locally for fast filtering / reporting (only group DNs, user names, databse id's and membership info as id-id pairs). I only need the memberOf attribute of users and groups, recursive loops are done locally. It will work for all LDAP implementations.

    What do you guys think? I'm not a network admin, but a programmer, so I'm no expert in network setups and when to use AD DS or AD LDS. The thing is I want to use this code at different customers without knowing their network setup (except for the domain name(s), LDAP host/port and bind user to connect to LDAP).

    Thanks a lot!

    Paul



    • Edited by Paul ML Saturday, March 29, 2014 9:44 AM
    Saturday, March 29, 2014 9:39 AM

Answers

  • I want to write a tool that can answer questions like "what users from group ABC have delete permission in all the (sub)directories of server MyDataServer?". This results in a list of directories and users and includes nested group membership. So it's about effective permissions. That's why I want all information in a SQL database so I can answer these questions with a single query in milliseconds. Otherwise, in order to answer these questions, I would have to get all members from group ABC and determine the nested groups for all these members (which can be thousands) for every report. Using a SQL database I can retrieve this information once a night for all the members.

    But I guess I will use the LDAP_MATCHING_RULE_IN_CHAIN syntax which gives me all nested groups for a member and should work for all AD installations from W2K3 SP2 and higher. When I want to support other LDAPs I will use another method for that specific LDAP.

    Sunday, March 30, 2014 8:06 AM

All replies

  • What have you tried?  If you just have questions on how to use AD then post here:

    http://social.technet.microsoft.com/Forums/windowsserver/en-US/home?forum=winserverDS

    I think you need to learn how to use AD before attempting to do this.  Also, this is a scripting forum and not a programming forum. If you have issues with a program then you need to post in the forum for that tool or language.

    If you look in the Repository/Gallery linked on the upper menu you will find numerous scripts that enumerate nested groups.  YOu can review those as they will show you how this is done in a script.

    Your shopping list above is overthinking a simple task that you will understand once you learn how to use AD.  I would start by reading the Microsoft documentation on AD.

    http://msdn.microsoft.com/en-us/library/aa772217(v=vs.85).aspx


    ¯\_(ツ)_/¯

    Saturday, March 29, 2014 2:58 PM
  • Thanks for the reply. I will post my question in the Directory Service forum. I used this forum because several google hits on 'nested groups' brought me to this forum. My question is not about the program, it can either be a script, doesn't matter.

    I know the scripts that retrieve nested groups, but almost all of them use recursive loops based on the memberOf attribute. So first all the groups of a user are retrieved using the memberOf attribute, then for all those groups they retrieve the memberOf attribute etc.. in order to get all the nested groups for a certain user. That is very bad for the performance and executes a huge number of queries to the AD, especially because I want to retrieve that information for all the users in the domain or given OU.

    I have used the ADSI interface before, but I don't want to bind the program to COM components. Its written in Java and can also run on Linux for example. Then again, also with ADSI you need to make the decision on how to retrieve the nested groups. It is not a simple task, because as you can see every choice has (dis)advantages and side effects. And I want to use an approach that will work in all network environments.


    • Edited by Paul ML Saturday, March 29, 2014 7:18 PM
    Saturday, March 29, 2014 7:16 PM
  • Sorry but I think you need to spend some time learning what LDAP and AD are.

    I am a programmer and have programmed AD since its beta. 

    AD is NOT a relational database.  It is an object repository.  Objects are stored and indexed for access. LDAP is the query language.  Enumeration of children is the only method of retrieving collections and recursively enumerating a group is the only way to get all of its members.YOu can go the other way.. You can get an object and recurse through the "memberOf" attribute to see its "belongingness"

    That is it.  There is not other magical API to get you what you want.  Extracting everything from AD into a SQL database would be a disaster because you would almost never be up to date without a complete refresh before each use.

    AD is very efficient.  You do not need to worry about performance.  AD performs well with 10s of thousands of users. All users are querying AD quite frequently when logged in.  For LDAP queries AD is much faster than a SQL database which is why Microsoft and most other companies did not use a SQL engine for their directory services repository.

    Here is a pretty good book that wil help you to understand the how and what of AD: http://www.sapien.com/books/Managing-Active-Directory

    There are a number of others that are more technical but I would start with this one as it is functional and has many examples of how to use AD in a scripted environment.


    ¯\_(ツ)_/¯

    Saturday, March 29, 2014 7:56 PM
  • I would say this.  If you insist on Java which is a very poor per forming language on most platforms then you can either bind Java to Net or purchase a third party cross-platform library.  In the end all will use COM on Windows and whatever LDP API is native to other platforms.

    If you search you will find dozens of LDP tools that work cross platform.  Many of these are free and have available source code.

    There are many Java LDP tools also that run on Unix and Windows.  They all use Java  over COM (ActiveX) on Windows. I don't think anyone is doing Java over Corba.

    Look into Mac OS-X tools for Windows to see what they are using for Mac clients attached to an AD forest.  MAc and Unix mostly use OpenLDAP.  OpenLDAP also runs on Windows.  There are Java client solutions for OpenLDAP which may be what you are looking for.


    ¯\_(ツ)_/¯

    Saturday, March 29, 2014 8:04 PM
  • Another problem I am having with your question is that it is a bit convoluted because it is not a programming , scripting or how to quesiotn. It is a basic issue about how to use something that is badly defined by your question.

    LDAP is a protocol. In Windows we show it like this: "LDAP://" THis LDAP: is a device access ID like HTTP:, FTP:, SMTP:, LPT1: or COM1:. All of thise are mapped int a device table to access a device handler. What comes after the colon is usually a command or query string.  The format is specific to the device handler.  THe article below shows you how to use "LDAP://" with a browser to access a DS.

    To see how this is defined as a network standard you need to start by reading this simple definition of LDAP.

    http://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol

    These are the commands that work across all implementations of DS on any platform that supports LDAP.  With SMTP we can use telnet to access the server and type it's commands. With this info you can pretty much do a query with a secure telnet session by typing strings although I am pretty sure that most implementations do not allow this.  You can use a simple TCP API to query any DS implementation.


    ¯\_(ツ)_/¯

    Saturday, March 29, 2014 8:29 PM
  • Java supports LDAP access out of the box using JNDI (and just TCP/IP underneath), there is no need to purchase third party stuff. You say there is no magical API to get nested groups, apart from recursive loops, but as I mentioned you also have attributes like tokenGroups and tokenGroupsGlobalAndUniversal and the search syntax defined by RFC2254 (LDAP_MATCHING_RULE_IN_CHAIN). They all do one LDAP request to get all the nested groups, instead of multiple calls using recursion. The only problem is that they require certain network setup, like tokenGroupsGlobalAndUniversal is not available on AD LDS. And tokenGroups doesn't return local groups.

    The question is quite simple, I just want all the nested groups per user in a way that it works in all (AD) network environments. Recursion on memberOf attribute seems to me quite a performance killer, so I was hoping to get some more insight on these tokenGroup attributes.


    • Edited by Paul ML Saturday, March 29, 2014 9:09 PM
    Saturday, March 29, 2014 8:59 PM
  • If an LDAP server doesn't support tokenGroups, and if tokenGroups doesn't support local groups, then I don't see an alternative to a recursive search if you want maximum compatibility.


    -- Bill Stewart [Bill_Stewart]

    Saturday, March 29, 2014 10:30 PM
    Moderator
  • Java supports LDAP access out of the box using JNDI (and just TCP/IP underneath), there is no need to purchase third party stuff. You say there is no magical API to get nested groups, apart from recursive loops, but as I mentioned you also have attributes like tokenGroups and tokenGroupsGlobalAndUniversal and the search syntax defined by RFC2254 (LDAP_MATCHING_RULE_IN_CHAIN). They all do one LDAP request to get all the nested groups, instead of multiple calls using recursion. The only problem is that they require certain network setup, like tokenGroupsGlobalAndUniversal is not available on AD LDS. And tokenGroups doesn't return local groups.

    The question is quite simple, I just want all the nested groups per user in a way that it works in all (AD) network environments. Recursion on memberOf attribute seems to me quite a performance killer, so I was hoping to get some more insight on these tokenGroup attributes.


    So why didn't you ask that question initially?  Why spend so much time proposing these odd solutions?


    ¯\_(ツ)_/¯

    Sunday, March 30, 2014 12:17 AM
  • I would guess that all of us want to know why you are asking this.  What is it you are trying to accomplish?

    ¯\_(ツ)_/¯

    Sunday, March 30, 2014 12:19 AM
  • I want to write a tool that can answer questions like "what users from group ABC have delete permission in all the (sub)directories of server MyDataServer?". This results in a list of directories and users and includes nested group membership. So it's about effective permissions. That's why I want all information in a SQL database so I can answer these questions with a single query in milliseconds. Otherwise, in order to answer these questions, I would have to get all members from group ABC and determine the nested groups for all these members (which can be thousands) for every report. Using a SQL database I can retrieve this information once a night for all the members.

    But I guess I will use the LDAP_MATCHING_RULE_IN_CHAIN syntax which gives me all nested groups for a member and should work for all AD installations from W2K3 SP2 and higher. When I want to support other LDAPs I will use another method for that specific LDAP.

    Sunday, March 30, 2014 8:06 AM
  • Impossible.  Permissions are not kept in Active Directory.  AAD knows nothing about file or object security.  LDAP has noting to do with this.

    There is a tool called Effective Permissions Wizard.  It can tell you what rights any user has on any object.  It does this by knowing how to validate in the security manager using the security API.  It is direct and immediate.

    I recommend that you first learn what and how Windows works and how Windows security works.  You will find that your tool is already built into Windows.

    There are many good books on Windows technical design.  I always recommend that anyone wanting to write Windows programs that are more than just data entry forms should study one of these books. Trying to guess your way through how Windows works will never succeed. It is not intuitive and it does not work like most other operating systems. If anything it is similar internally to DEC VMS 5 because Windows NT was designed by one of the engineers for VMS from DEC. Knowing Unix or Novell is of little use outside of the command line.

    Start with one of the many books on Windows internals.


    ¯\_(ツ)_/¯

    Sunday, March 30, 2014 8:23 AM
  • Nothing is impossible ;) I already extract ACL information from the directories into a SQL database. I know the effective permission tab in Windows, but it only shows me the effective permissions for one member and one particular folder at the time. This is completely useless if I want to check the effective NTFS permissions for a group of thousands of users for all the directories on the server (including nested group membership). I want to display the security flaws, for example users that have 'accidentally' delete or even read permission on certain folders.

    • Edited by Paul ML Sunday, March 30, 2014 12:04 PM
    Sunday, March 30, 2014 12:02 PM
  • If you have already done this then why are you asking how to do it? Why are you asking about using Java when it is not native to Windows and is a very poor performer.

    There are many tools already available that can monitor and report permissions across an enterprise.  Why do you need to build one? 

    The questions you ask and the statements you make are nearly identical to those someone asked about a year ago.  The

    y gave up after thinking this through and after learning how Windows AD and security work.

    In a large organization these things are not static so placing them in a database will guarantee that you will never be current with the information.

    There is never a need to know the exact mapping.  We only  need to verify group application.  This can be enforced with Group Policy.  We only need to know which groups users are assigned to.  Group design is critical to maintenance and good security.  Once properly implemented we can use any of a number of security reporting tools to report compliance to policy.

    Based on this what is your question? 


    ¯\_(ツ)_/¯

    Sunday, March 30, 2014 12:16 PM
  • I want to write a tool that can answer questions like "what users from group ABC have delete permission in all the (sub)directories of server MyDataServer?". This results in a list of directories and users and includes nested group membership. So it's about effective permissions. That's why I want all information in a SQL database so I can answer these questions with a single query in milliseconds. Otherwise, in order to answer these questions, I would have to get all members from group ABC and determine the nested groups for all these members (which can be thousands) for every report. Using a SQL database I can retrieve this information once a night for all the members.

    But I guess I will use the LDAP_MATCHING_RULE_IN_CHAIN syntax which gives me all nested groups for a member and should work for all AD installations from W2K3 SP2 and higher. When I want to support other LDAPs I will use another method for that specific LDAP.

    Again - note that this question has nothing to do with LDAP or AD.  It just asks what group has permissions on what resources.

    I really think you would do well to spend time understanding the NTFS and its security along with how we sue security in Windows.  By assuming this has something to do with AD you are making it a bigger issue than needed.  AD is a repository for accounts and trusts and manages authentication and security group membership.  All file security is managed by the OS that hosts the files and not by AD.  Users are not normally granted access to resources through direct inclusion in the DACL but are given access through membership in one or more groups.  Loading AD into a SQLL database will not help you.


    ¯\_(ツ)_/¯

    Wednesday, April 2, 2014 3:16 PM