none
Unable to see Active Directory Groups in the User Profile Database after Profile Import

    Question

  • ***Major Update - I have finally been able to get the direct attention of the folks responsible for the User Profile Service on the SharePoint Product Team.  Long story short, they have reproduced the error and identified as an actual mistake that needs to be fixed, so it is now officially in the bug pipeline and will be fixed.  The current estimate is some time in the summer.  They will keep me updated with timeframes, which I am allowed to share as time goes on.

    SharePoint Server 2010 Enterprise RTM. W2K8R2 w/multi-server setup:

    • AD/DNS
    • SQL 2008
    • WFE
    • APP
    • Claims Mode Web App only using Windows Integrated Auth

    So, this was never a problem in 2007, and I didn't even realize it was a problem in 2010 until I started to build a solution that utilized my blog article: InfoPath - User Roles in Browser-Enabled Forms Using AD Groups.  I went to utilize the same web method of the same web service, but I noticed that no data was showing up at all.  Typically, the GetUserMembership/GetCommonMembership methods return the specified user's memberships: AD Security Groups, AD Distribution Lists, and SharePoint Sites (not SharePoint Groups, though).

    • My user profile sync is working.  All AD users are pulled in with the proper profile data.
    • "Users and Groups" is selected in the Synchronization Entities section of my Sync Settings.
    • Security groups are working for permissions and audience targeting.  Confirmed my users are affected properly by the use of Security Groups.
    • My query to the GetUserMemberships web method (and GetCommonMemberships) is running (not failing), but it's not returning anything even though my user is in some Security Groups and has explicit membership to multiple sites.
    • The GetUserProfileByName method of the same UserProfileService.asmx web service returns all the regular profile data like expected, so the web service works and my profile database is populated

    Basically, I'm not seeing my AD groups or any membership data populated in the profile database.  I did use MIISCLIENT.exe to see what I could find, and here is what I saw:

    • Using the Metaverse Search, I searched for the "person" type and saw all of the users in my profile sync connection (single OU)
    • Using the same tool, I searched for the "group" type and saw nothing, but the message said 4 items were retrieved
    • I realized that the only column showing was displayName, and they were blank, so I added other columns to be sure
    • objectGUID, objectType, distinguishedName all showed values, and I could now see all the Security Groups from the OU where I'm doing my profile sync
    • My "person" objects all have displayNames showing but none of the groups do.  In SharePoint, the GetUserMemberships method relies on displayName and accountName, but neither are coming through the profile import

    So, it does seem like the groups are coming in with the profile import, but I can't see them.  I also can't verify that the groups are being associated with my users in the profile database, because doing a query to the membership methods returns nothing...not even blank rows.

    ***Edit:  New information!  Regular AD Distribution Lists _do_ work properly.  I just never bothered testing them until folks on my blog notified me.  DLs come through the profile sync, are visible in the profile database, and show up when using the GetUserMemberships method.

    ***Edit: Ok, now we're getting somewhere.  I checked my last profile sync with the MIISCLIENT, and this is what I found:

    Here are the properties of my Distribution List:

    Here are the properties of my Security Group:

    Notice that the groupType value of the DL is a normal integer (2), but the groupType value of the SG is some crazy negative number.  Both types are still lacking DisplayNames for some reason, but when I retrieve the DL via GetUserMemberships, it DOES show the proper DisplayName despite nothing showing in the MIISCLIENT.


    SharePoint Architect || Microsoft MVP || My Blog

    • Edited by Clayton Cobb Saturday, February 04, 2012 4:20 AM
    Friday, June 18, 2010 8:25 AM

All replies

  • Dear Clayton

    Your question falls into the paid support category which requires a more in-depth level of support.  Please visit the below link to see the various paid support options that are available to better meet your needs. http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone

     


    Regards, Savoeurn Va Microsoft Online Community Support
    Monday, June 21, 2010 8:29 PM
  • Really?  Why is that?  This appears to be a systematic problem, not a localized problem.  What makes it require paid support?  This is one of many things I've run into while testing, and everything until now was answered by the InfoPath team directly or through them reaching out to other product teams.  No one so far has been able to answer this, which means no one has confirmed that it DOES work anywhere for anyone.  Is it typical to have to pay for support related to a systematic bug/issue?
    SharePoint Architect || Microsoft MVP || My Blog
    Monday, June 21, 2010 8:56 PM
  • Hi Clayton,

    Customer satisfaction and accurate resolution is very important to us.  With that said, I was following the steps outlined in post and I have determined that we have do a much deeper level of investigation to provide a proper resolution. 

     


    Regards, Savoeurn Va Microsoft Online Community Support
    Wednesday, June 23, 2010 7:40 PM
  • Hi Clayton,

    Customer satisfaction and accurate resolution is very important to us.  With that said, I was following the steps outlined in post and I have determined that we have do a much deeper level of investigation to provide a proper resolution. 


    Regards, Savoeurn Va Microsoft Online Community Support

    I definitely believe that, but I didn't think I would have to pay to have MSFT look at their own bug.  I've spent countless hours testing and submitting bugs for SP2010 in my own free time since last July, so I was very disappointed to hear that after all that work, I now have to pay to have you fix a bug that I've found.  Is this how it works?
    SharePoint Architect || Microsoft MVP || My Blog
    Wednesday, June 23, 2010 7:54 PM
  • Hi Clayton,

    If it is determined to be a bug, then it doesn't cost anything. I hope this clears up your concerns.


    Regards, Savoeurn Va Microsoft Online Community Support
    Wednesday, June 23, 2010 9:38 PM
  • Hi Clayton,

    If it is determined to be a bug, then it doesn't cost anything. I hope this clears up your concerns.


    Regards, Savoeurn Va Microsoft Online Community Support


    I guess that's a real eye-opener.  I suppose the issue will sit and no one in the world will be able to use it, and they simply won't know why.  It's hard to describe how disappointed I am to see this response after all the time I've put in for free to help make this a better product.

    I will not be calling paid support.


    SharePoint Architect || Microsoft MVP || My Blog
    Wednesday, June 23, 2010 11:13 PM
  • I too have found this to be a problem whilst I was trying to follow Claytons excellent example of implementing a user roles based solutions within InfoPath browser forms; something which we are / were hoping to roll out for our large scale enterprise document forms.

    As the same UserProfile service retrieves and populates the data for My Sites it has been noted by some of our users that the memberships tab in My Sites is not being populated; it would therefore seem that this "must" be a core feature of SharePoint 2010 as the same functionality was available previously in MOSS.

    I hope that a solution is found to this sooner rather than later.

    Thursday, July 08, 2010 3:24 PM
  • Hi

    I also am unable to see ou group 'display name' using MIISCLIENT. 

    I wondered if it was possible to re-map the property and run the import again but that didnt seem to do anything.

    I thought this issue might help me resolve my question here: http://social.technet.microsoft.com/Forums/en-US/sharepoint2010setup/thread/586494b9-d259-4abf-a857-26137fa30460 but no luck so far.

    Ash


    Fast, Cheep, Good. Choose any Two!
    Monday, August 02, 2010 1:29 PM
  • Maybe you can make the paid support call to Microsoft, since they have decided not to help us for some reason.

    I still have not found one person who can confirm that this IS working in their 2010 environment.


    SharePoint Architect || Microsoft MVP || My Blog
    Monday, August 02, 2010 1:56 PM
  • Actually the whole "you need to phone CSS" response above is ridiculous. This is a common problem with UPS configuration with SP2010. It *is* possible to import groups. It's even more ridiculous to suggest that approach as the first response. Extremely disapointing.

    So let's try a more appropriate approach to this problem.

    Clayton, can you provide me (over email perhaps) a ULS log filtered on Catagory = User Profiles for when you perform an incremental sync? I will then work with you to resolve this, or determine if you are hitting an issue caused by a "known" problem.

    s.


    Cheers
    Spence
    www.harbar.net
    Microsoft Certified Master | SharePoint 2007
    Sunday, August 08, 2010 6:49 PM
  • Thanks, Spence.  Heading on vacation today, but I will get that done and send it to you.  I've had the InfoPath team working on it, but they haven't had luck yet, mostly due to it not really being an InfoPath problem.  Thanks for the assistance...
    SharePoint Architect || Microsoft MVP || My Blog
    Sunday, August 08, 2010 6:52 PM
  • Hi Spence

     

    Firstly, thanks for all the work around Kerberos for SP 2007 on your site, helped me a great deal.

    Now the new product is out and the focus changes again :)

    Can you have a quick look at http://social.technet.microsoft.com/Forums/en-US/sharepoint2010setup/thread/586494b9-d259-4abf-a857-26137fa30460 which is on the same lines as this topic of Claytons.

    I was wondering if you could explain the difference between selecting audience membership, and selecting say, members/permissions into a document library.  As i say in my post, the 'picker' for audience membership is empty, with no users or groups showing, but on configuring permissions for anything else, all users and groups are present.  Are these looking at two different data sources then? 

    Thanks for your time.

    Asher


    Fast, Cheep, Good. Choose any Two!
    Monday, August 09, 2010 10:22 AM
  • Hi,

    I am also in the same situation with Clayton, I can't reach a security group with Sharepoint 2010 - Usersync. I tried everything to get it work but I think it is clear that there something really wrong with SP2010. I really am very dissapointed with that.

    Hope Mirosoft find a solution for that.

     

    Regards

    Friday, August 27, 2010 7:37 AM
  • Spence, I did the work and got you some logs, but there isn't much to them.  I sent you both a Full Sync and Incremental Sync, but the logs are identical for both other than the timestamps/IDs.  No errors - just the regular progression as far as I can tell, but hopefully you see something missing or out of sorts.

    Btw, after quite a battle, the InfoPath team convinced the support team to open a case for me without charging, so I do have an engineer on it now.  He couldn't get groups initially either but said he has to build a new, clean system to verify that his testing is precise, so he'll be trying to repro my issue and get it figured out.

    I'd be interested in hearing what this "known" problem is that's related to retrieving AD groups.

    Thanks!


    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Saturday, August 28, 2010 8:27 PM
  • I have another piece of information that I'm going to also add to the original post.  Thanks to the diligent effort of people on my blog who all want to use this functionality, we have found that regular AD Distribution Lists _do_ come through properly and _do_ show up when invoking the GetUserMembership method of the UserProfileService.  I had not bothered testing DLs, because I never use them, and that's due to the fact they can't be leveraged for permissions in SharePoint.  However, I've always known they come through the profile import specifically because of how much I work with the UserProfileService.  They've always been there for me to use in 2007, but I never really messed with them.

    So, I created a DL in my environment, added my users, did an incremental profile sync, and then tested my form again.  Right away, the new DL showed up with all of its information available via the GetUserMembership method.  I'm now going to inspect the profile sync traffic with the MIISCLIENT to see if the behavior of the DL is different than the SGs.  Previously, I saw my SGs coming through with blank DisplayName fields, which meant I didn't even noticed them until I did some searching and saw that I had 4 "items" despite seeing no names.  I later added the proper columns to see that 4 SGs had come through but had no names.  I'm betting the DL will have a DisplayName...


    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Saturday, September 18, 2010 4:25 PM
  • It seems like it comes down to profile service and FIM is the culprit here not bringing the AD SG to the sharepoint. Is MS planning to fix this issue in future patches or service packs?

    As we know, User Profile Services and FIM has many issues. I do understand the reason why MS has adopted to use FIM for the user profile sync but I really wish MS has gave us two profile sync options like two authentications (classic and claim) - one profile sync can be classic way like MOSS 2007 where you import only and other one with FIM to support both profile import/export. In that case, one can still use the MOSS 2007 way of profile sync and we wouldn't have this issue at all.

    Clayton, Hope to see MS fix  this issue. Thanks for such a great work and hopefully MS does this free for us. :)

    Nik

    Saturday, September 18, 2010 7:50 PM
  • I actually do have a case open, but they haven't really done much.  It took a month for them just to reproduce the error after the case was opened.
    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Saturday, September 18, 2010 7:53 PM
  • You will only get a DisplayName back if that attribute in AD is present. It's a direct mapping between the two fields.
    Cheers
    Spence
    www.harbar.net
    Microsoft Certified Master | SharePoint 2010
    Microsoft Certified Master | SharePoint 2007
    Monday, September 20, 2010 9:12 PM
  • You will only get a DisplayName back if that attribute in AD is present. It's a direct mapping between the two fields.
    Cheers
    Spence
    www.harbar.net
    Microsoft Certified Master | SharePoint 2010
    Microsoft Certified Master | SharePoint 2007

    In my case, the DisplayName is present in AD, or at least I thought it was, but the field is actually named "Description," and that value DOES show up in MIISCLIENT as you can see above.  I don't actually see a DisplayName field in AD for groups - there is just a Group Name and Description - but then MIISCLIENT makes it appear as if nothing exists, because it defaults to showing Display Name even for groups.  Just an unfortunate combination?
    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Monday, September 20, 2010 9:22 PM
  • Yes, it's just a symptom of how the Metaverse view is configured by default.

    DisplayName is present on objects of class Group, but they are not displayed in ADUC. To edit this value you need to use the Advanced Features view or ADSIEdit.msc. by default the SharePoint MA maps the DisplayName to the profile db but of course it wil be empty by default.

     


    Cheers
    Spence
    www.harbar.net
    Microsoft Certified Master | SharePoint 2010
    Microsoft Certified Master | SharePoint 2007
    Monday, September 20, 2010 11:47 PM
  • Yes, it's just a symptom of how the Metaverse view is configured by default.

    DisplayName is present on objects of class Group, but they are not displayed in ADUC. To edit this value you need to use the Advanced Features view or ADSIEdit.msc. by default the SharePoint MA maps the DisplayName to the profile db but of course it wil be empty by default.

    Hi,

    We are experiencing the same problems listed here on our SharePoint intranet instance.
    Is it alright if I interject for a bit here to ask for some clarification?

    The SharePoint MA (Management Agent Designer in FIM Client?) on our server is listing what you stated above. displayName is being mapped to a Metaverse Attribute named displayName, and because displayName isn't normally populated in AD on a security group, it causes a lookup for the group by the display name to fail correct?

    The possible fixes for this would be to apply the attribute using the advanced features view in the AD tool or use ADSIEdit.msc to fill it in... OR use the FIM Client to map the description in the object type "group" to displayName in the Metaverse?

    Thanks, Tim

    Tuesday, September 28, 2010 5:33 PM
  • Here is what one of our clients is experiencing, related i believe to what is discussed here ..

    We are creating audiences by using the user is a "member of" a security group in AD.   When we add a member to the AD group directly from users and computers in AD, the audience is not updated after the addition and a compilation run of that associated audience occurs.   The only way the newly added user shows up in the compilation is when we run a full profile import and follow that up with a re-compilation of that audience.   Not a good option if the AD store is large, that being the case here.  

    Looking to the better kind for some thoughts ..

     

    Sunday, October 24, 2010 4:23 AM
  • I would create a new thread for the audiences, because this thread is very specific and has a ticket open with Microsoft.  We may find that the issues are related, but there has been no association made so far in the troubleshooting.
    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Sunday, October 24, 2010 7:36 AM
  • Got a response from Microsoft...

    The issue is confirmed and recognized.  The design change _may_ (not for sure) be included in SP2.  Since that's a long way out, and it's not 100% certain, my only recommendation is to build your own Active Directory Web Service for retrieving this type of info, or use the Qdabra Active Directory Web Service, which is only $299 and gets all the user profile data I need plus a lot more, so I'm now using this in my 2010 environments and even in some 2007 environments to replace Contact Selectors in InfoPath that don't work in non-IE browsers due to ActiveX.

    Unfortunately, I have no workaround for how this affects audiences, but I did vehemently plead with the MSFT engineer to not close the ticket and instead ask the product team for an answer to how we are supposed to work around this issue for audiences.


    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Wednesday, December 22, 2010 10:43 PM
  • Hello Clayton. Have you heard anything new from Microsoft about this issue?

    Monday, November 21, 2011 11:00 AM
  • Hi

    does anyone know if Microsoft issued a fix for this yet? or if they will soon or newar future?

    This bug is really crippling

    Wednesday, January 25, 2012 2:31 PM
  • Hi

    does anyone know if Microsoft issued a fix for this yet? or if they will soon or newar future?

    This bug is really crippling


    I've heard nothing.  I'll ask the InfoPath team for an update.
    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Thursday, January 26, 2012 8:46 AM
  • Hi

    does anyone know if Microsoft issued a fix for this yet? or if they will soon or newar future?

    This bug is really crippling


    I've heard nothing.  I'll ask the InfoPath team for an update.
    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force


    Thanks to the hard work by Laura Harrington on the InfoPath team, we were finally able to get this resolved!

    ***Major Update - I have finally been able to get the direct attention of the folks responsible for the User Profile Service on the SharePoint Product Team. Long story short, they have reproduced the error and identified as an actual mistake that needs to be fixed, so it is now officially in the bug pipeline and will be fixed. The current estimate is some time in the summer. They will keep me updated with timeframes, which I am allowed to share as time goes on.


    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force
    Saturday, February 04, 2012 4:20 AM
  • I've also hit this one!

    Is there a bug/issue number so I can add a Google alert for any hotfixes etc? No resolution since June 18 2010 is pretty astounding.

    Thanks for the thorough investigation Clayton - was starting to fall down too many rabbit holes looking for a non-existent problem in my environment...

    Thursday, March 15, 2012 1:54 AM
  • Thank you for the update, Clayton. I was so excited to see your original blog about switching/displaying views based on user security and was hoping it would work in SP2010....so utterly disappointed that it is a bug and this solution will not be available at this time. It is such a needed business element, especially when it comes to filling out and approving InfoPath forms with workflow. It just seems like user based security would be a no brainer, especially if it worked in 2007!!!

    *very frustrated*

    Tuesday, March 20, 2012 7:20 PM
  • DGoss04, at least it is now a recognized issue that is being addressed and will be fixed.  I just don't know how long it will take.

    SharePoint Architect || Microsoft MVP || My Blog
    Planet Technologies || SharePoint Task Force

    Tuesday, March 20, 2012 7:23 PM
  • Something about crazy negative number Object: group ~ Attribute: groupType

    Мнения, высказанные здесь, являются отражением моих личных взглядов, а не позиции корпорации Microsoft. Вся информация предоставляется "как есть" без каких-либо гарантий.

    Wednesday, June 13, 2012 8:29 AM
  • Any updates to this?
    Monday, July 30, 2012 9:26 PM
  • Crazy that you have to put forth so much effort to get their attention Clayton when clearly you are able to reproduce the issue and are well known in this forum.  Your posts and blog have been a HUGE help for an Infopath beginner like myself.  I appreciate all the time you've put into your blog and keeping the community up to date.

    For now I will maintain a manual list in my small form but ideally I'd do an AD group query as you were able to do in 2007. 

    Just wanted to say 'Thanks', and subscribe to this thread for updates.

    -Chris


    • Edited by cmille34 Tuesday, July 31, 2012 2:55 PM Spelling error.. durr
    Tuesday, July 31, 2012 2:54 PM
  • I recently came across this in Office 365 when I ran the readiness checklist and it told me that my displaynames were missing from several of my AD groups.  I also used Hyena software to verify if my display names were missing and found that to be true.  So I am wondering if this has something to do with all that you are seeing as well in sharepoint where the groups are not displaying correctly.  I am trying to find out how to use the ADSIEdit tool to update the groups but this is still an ongoing lesson for me to figure out why the displayname was missing.  Somehow we seems to have lost this or maybe they were never there when I came on board.  I came across this article that I wanted to share with you on the display name issue.

    http://support.microsoft.com/kb/250455

    But I do not know if this will help with group names not being able to display correctly but it was something I came accross when trying to move my AD to the Cloud.

    my readiness error was just this:

    Discovered groups without a displayname
    Note: Groups without a displayname will NOT get synchronized to Office 365.
    Updating the displayname attribute in Active Directory for each group will resolve this issue when DirSync is enabled.


    WorkerBee09

    Wednesday, September 19, 2012 1:23 PM
  • Has anyone tested the Aug2012 cumulative update to see if this helps or hurts the situation?

    Just wondering when that fix is coming out!!

    Thursday, October 25, 2012 7:46 PM
  • Have not heard anyone that has sovled this short of forcing the display names using ADSI. Has this been fixed by MS? Have really been suffering not being able to use group membership in forms or or for audience compilation.

    I list this as another failure by MS. The first was the Managed Metadata in InfoPath forms and lacking a way to search using webparts by metadata.

    Monday, December 03, 2012 11:00 PM
  • Have not heard anyone that has sovled this short of forcing the display names using ADSI. Has this been fixed by MS? Have really been suffering not being able to use group membership in forms or or for audience compilation.

    I list this as another failure by MS. The first was the Managed Metadata in InfoPath forms and lacking a way to search using webparts by metadata.

    Since our migration from SharePoint 2007 to SharePoint 2010, we have had several problems similar to this.  I see this problem also on our production site, so there is no way Microsoft can say this is an isolated occurrence.  We found a problem with datasources and Infopath forms.  The problem maxxed the CPU's of our Front End server at 100%.  We reported it to Microsoft and the supposedly came up with a fix.  We appled the fix and the problem still occurred.  It took over a month for them to admit it was a problem with SP2010. In the mean time we had to change code that was working fine in SP2007, but would no longer work in SP2010.

    If an update is produced from Microsoft, will you keep everyong updated?  I think many of us in the SharePoint community have experienced the pain of working with Microsoft.  Clayton, you are not alone with the time and effort it takes to PROVE to Microsoft their products have bugs.  When are they going to start listening to their customers.

    Monday, March 18, 2013 1:33 PM
  • After almost 3 years still no fix...
    Tuesday, April 23, 2013 5:12 PM
  • Hello Clayton,

    I have same problem with SP 2010 and now I am testing SP 2013 and seems that the problem still.

    Probably I have some configuration problem, but all seems work, except this.

    Tuesday, June 11, 2013 2:42 PM
  • Hi Team,

    Any help received so far? Any updates to this issue? We too have the exact same trouble inside our SharePoint environment.

    Thanks and Regards,

    ~ Anubhav

    Thursday, November 21, 2013 6:41 PM