none
LDAP Paged Search Problem

    Question

  • Hi,

    I posted this question in the "Directory Services" forum but was told it was a wrong forum (http://social.technet.microsoft.com/Forums/en-CA/winserverDS/thread/478f03b8-9a15-44d9-81c3-c72ab786faf1), so if this is a wrong forum again, kindly direct me to the right one, thanks.

    I was trying to create a general LDAP Search function not using the DirectorySearcher class, after running into issues with it (http://dunnry.com/blog/PagingInSystemDirectoryServicesProtocols.aspx).  As suggested by several posts on the net, I'm using DirectoryServices.Protocols namespace to bypass the issues with the COM objects.  However, I don't seem to get the paging part to work.  The code only returns the first page of results.  More precisely, the PageResultResponseControl does not return a cookie, although many more pages are expected.

    Here is my code:

    function LDAPPagedSearch ([string]$searchRoot = ([adsi]'LDAP://rootDSE').defaultNamingContext.value, [string]$filter = {throw 'filter is required'}, [String[]]$attributes = @('name'), [switch]$typesOnly, $dc = $script:domainController)
    {
    	$resultSet = @()
    	$connection = New-Object System.DirectoryServices.Protocols.LdapConnection($dc)
    	$connection.Timeout = New-Object system.TimeSpan(3.8e10)
    	$searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($searchRoot,$filter,[System.DirectoryServices.Protocols.SearchScope]::Subtree,$attributes)
    	$searchRequest.TimeLimit = New-Object system.TimeSpan(3e9)
    	$searchRequest.SizeLimit = [int]::MaxValue
    	$searchRequest.TypesOnly = $typesOnly
    	$pageResultControl = New-Object System.DirectoryServices.Protocols.PageResultRequestControl(10)
    	[Void]$searchRequest.Controls.Add($pageResultControl)
    	
    	do
    	{
    		$searchResponse = [System.DirectoryServices.Protocols.SearchResponse] $connection.SendRequest($searchRequest)
    		$searchResponse.Controls | ? { $_ -is [System.DirectoryServices.Protocols.PageResultResponseControl] } | % {
    			$pageResultControl.Cookie = ([System.DirectoryServices.Protocols.PageResultResponseControl]$_).Cookie
    		}
    		$resultSet += $searchResponse.Entries
    	}
    	while ($pageResultControl.Cookie.length -gt 0)
    	
    	return $resultSet
    }
    
    cls
    $script:domainController = ([ADSI]"LDAP://RootDSE").dnsHostName.value
    $searchRoot = ([adsi]'LDAP://rootDSE').defaultNamingContext.value
    $searchFilter = '(&(objectcategory=person)(objectclass=user))'
    $searchAttrib = @('name')
    
    $usersResult = @(LDAPPagedSearch -searchRoot $searchRoot -filter $searchFilter -attributes $searchAttrib -typesOnly)
    
    
    


    So when the above code is run, I only get back 10 results (the first page), although there are 400,000 user objects in the domain.  Is there anything I did wrong?

    Thanks in advance.

    Saturday, September 24, 2011 4:05 AM

Answers

  • I got this similar code to work for me:

     

    [System.Reflection.assembly]::LoadWithPartialName("system.directoryservices.protocols") | Out-Null
    [int]$pageSize = 200
    [int]$count = 0
    $D = [System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()
    $Domain = [ADSI]"LDAP://$D"

    $connection = New-Object System.DirectoryServices.Protocols.LdapConnection($d)
    $subtree = [System.DirectoryServices.Protocols.SearchScope]"Subtree"
    $filter = "(&(objectCategory=person)(objectclass=user))"
    $searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($Domain.distinguishedName,$filter,$subtree,@("1.1"))
    $pagedRequest = New-Object System.DirectoryServices.Protocols.PageResultRequestControl($pageSize)
    $searchRequest.Controls.add($pagedRequest) | out-null
    $searchOptions = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
    $searchRequest.Controls.Add($searchOptions) | out-null

    while ($true)
    {
        $searchResponse = $connection.SendRequest($searchRequest)
        $pageResponse = $searchResponse.Controls[0]
        $count = $count + $searchResponse.entries.count
        # display the entries within this page.
        foreach($entry in $searchResponse.entries){$entry.DistinguishedName}
        # Check if there are more pages.
        if ($pageResponse.Cookie.Length -eq 0){write-Host $count;break}
        $pagedRequest.Cookie = $pageResponse.Cookie
    }

    -----

     


    Richard Mueller - MVP Directory Services
    Saturday, September 24, 2011 4:46 PM
  • Hi,
    Your script is ok - almost. You need only add one more controls like in Richard script:

    $SearchOptionsControl = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
    [Void]$searchRequest.Controls.Add($SearchOptionsControl)
     
    
    #correct function
    function LDAPPagedSearch ([string]$searchRoot = ([adsi]'LDAP://rootDSE').defaultNamingContext.value, [string]$filter = {throw 'filter is required'}, [String[]]$attributes = @('name'), [switch]$typesOnly, $dc = $script:domainController)
    {
     $resultSet = @()
     $connection = New-Object System.DirectoryServices.Protocols.LdapConnection($dc)
     $connection.Timeout = New-Object system.TimeSpan(3.8e10)
     $searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($searchRoot,$filter,[System.DirectoryServices.Protocols.SearchScope]::Subtree,$attributes)
     $searchRequest.TimeLimit = New-Object system.TimeSpan(3e9)
     $searchRequest.SizeLimit = [int]::MaxValue
     $searchRequest.TypesOnly = $typesOnly
     $pageResultControl = New-Object System.DirectoryServices.Protocols.PageResultRequestControl(10)
     [Void]$searchRequest.Controls.Add($pageResultControl)
     
     $SearchOptionsControl = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
     [Void]$searchRequest.Controls.Add($SearchOptionsControl) 
     
     do
     {
      $searchResponse = [System.DirectoryServices.Protocols.SearchResponse] $connection.SendRequest($searchRequest)
      $searchResponse.Controls | ? { $_ -is [System.DirectoryServices.Protocols.PageResultResponseControl] } | % {
       $pageResultControl.Cookie = ([System.DirectoryServices.Protocols.PageResultResponseControl]$_).Cookie
      }
      $resultSet += $searchResponse.Entries
     }
     while ($pageResultControl.Cookie.length -gt 0)
     
     return $resultSet
    }
    
    

    Saturday, September 24, 2011 5:17 PM

All replies

  • I got this similar code to work for me:

     

    [System.Reflection.assembly]::LoadWithPartialName("system.directoryservices.protocols") | Out-Null
    [int]$pageSize = 200
    [int]$count = 0
    $D = [System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()
    $Domain = [ADSI]"LDAP://$D"

    $connection = New-Object System.DirectoryServices.Protocols.LdapConnection($d)
    $subtree = [System.DirectoryServices.Protocols.SearchScope]"Subtree"
    $filter = "(&(objectCategory=person)(objectclass=user))"
    $searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($Domain.distinguishedName,$filter,$subtree,@("1.1"))
    $pagedRequest = New-Object System.DirectoryServices.Protocols.PageResultRequestControl($pageSize)
    $searchRequest.Controls.add($pagedRequest) | out-null
    $searchOptions = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
    $searchRequest.Controls.Add($searchOptions) | out-null

    while ($true)
    {
        $searchResponse = $connection.SendRequest($searchRequest)
        $pageResponse = $searchResponse.Controls[0]
        $count = $count + $searchResponse.entries.count
        # display the entries within this page.
        foreach($entry in $searchResponse.entries){$entry.DistinguishedName}
        # Check if there are more pages.
        if ($pageResponse.Cookie.Length -eq 0){write-Host $count;break}
        $pagedRequest.Cookie = $pageResponse.Cookie
    }

    -----

     


    Richard Mueller - MVP Directory Services
    Saturday, September 24, 2011 4:46 PM
  • Hi,
    Your script is ok - almost. You need only add one more controls like in Richard script:

    $SearchOptionsControl = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
    [Void]$searchRequest.Controls.Add($SearchOptionsControl)
     
    
    #correct function
    function LDAPPagedSearch ([string]$searchRoot = ([adsi]'LDAP://rootDSE').defaultNamingContext.value, [string]$filter = {throw 'filter is required'}, [String[]]$attributes = @('name'), [switch]$typesOnly, $dc = $script:domainController)
    {
     $resultSet = @()
     $connection = New-Object System.DirectoryServices.Protocols.LdapConnection($dc)
     $connection.Timeout = New-Object system.TimeSpan(3.8e10)
     $searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($searchRoot,$filter,[System.DirectoryServices.Protocols.SearchScope]::Subtree,$attributes)
     $searchRequest.TimeLimit = New-Object system.TimeSpan(3e9)
     $searchRequest.SizeLimit = [int]::MaxValue
     $searchRequest.TypesOnly = $typesOnly
     $pageResultControl = New-Object System.DirectoryServices.Protocols.PageResultRequestControl(10)
     [Void]$searchRequest.Controls.Add($pageResultControl)
     
     $SearchOptionsControl = new-object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
     [Void]$searchRequest.Controls.Add($SearchOptionsControl) 
     
     do
     {
      $searchResponse = [System.DirectoryServices.Protocols.SearchResponse] $connection.SendRequest($searchRequest)
      $searchResponse.Controls | ? { $_ -is [System.DirectoryServices.Protocols.PageResultResponseControl] } | % {
       $pageResultControl.Cookie = ([System.DirectoryServices.Protocols.PageResultResponseControl]$_).Cookie
      }
      $resultSet += $searchResponse.Entries
     }
     while ($pageResultControl.Cookie.length -gt 0)
     
     return $resultSet
    }
    
    

    Saturday, September 24, 2011 5:17 PM
  • Thanks Richard.  Will test it out on Monday at work.

    BTW an off topic question.  What does the @("1.1") stand for?  I know that is supposed to be a list of attributes but perhaps "1.1" is some sort of code for some AD attribute?

    Furthermore, is there any reference/links on what these codes are in LDAP?

    Saturday, September 24, 2011 8:14 PM
  • Wonderful Michal.  Will test it out on Monday.

    Appreciate you help!

    Saturday, September 24, 2011 8:15 PM
  • Here is the article that helped me figure out the paging:

    http://bsonposh.com/archives/325

    Notice that @("1.1") is passed for the array of attribute names. I have no idea why. All documentation I can find indicates it should be a string array of lDAPDisplayNames. However, no matter what I use, I can only output distinguishedNames. Even with Michal's version, I only get distinguishedName. We are missing something.

     


    Richard Mueller - MVP Directory Services
    Sunday, September 25, 2011 12:22 AM
  • Thanks Michal!  Tested and it worked like a charm.
    Monday, September 26, 2011 2:22 PM
  • Hi Richard,

    I followed Michal's instruction and modified my code and it now works like a charm.  Although I still don't understand the "1.1" thing but from its behaviour in my tests it seems to be a way to tell the DS not to return any attribute - only the distinguished name is returned.

    From my observation, if you put in an array of attributes, it gives you back the distinguished name + the attributes.  If you put in @('1.1') as 'the' attribute, it only gives you the distinguished name, useful if you just want to confirm the existence of an object.

    However, I still cannot find anything on the net that has a good explanation about this "1.1" thing.......

    Appreciate your time.


    Monday, September 26, 2011 2:33 PM
  • Hello,

    "1.1" thing comes from RFC 4511, paragraph 4.5.1.8 and is a proper way how to tell LDAP server that you are not interested in any attributes of objects. Jiri

    "A list containing only the OID "1.1" indicates that no attributes are to be returned. If "1.1" is provided with other attributeSelector values, the "1.1" attributeSelector is ignored. This OID was chosen because it does not (and can not) correspond to any attribute in use."

    Jiri

    Monday, May 25, 2015 1:01 PM