none
Group Memberships not Flowing into Metaverse RRS feed

  • Question

  • Hello,

    I'm trying to figure out why the group member attributes in the CS are not flowing into the MV.  Here's what I have:

    An HR system running on SQL Server
    A staging database that extract data from the HR system
    The staging database has a table representing person object
    The stating database has a table representing person multi-valued attributes (i.e location, job code, etc)
    The staging database has a table representing group objects
    The staging database has a table representing group memberships (mult-valued)

    A SQLMA connected to the person and person multi tables
    A SQLMA connected to the group and group membership tables

    All group memberships are based on job codes and locations.  There are no approval process in place.  If they have this job code, they get certain groups.  That's all calculated in the staging database and the memberships are in the group membership table

    This system does connect to AD (and a few other things), but I'm not concerned with that, right now.

    I've read 100 articles on this, most of them over 5 years old, and tried the ones that made sense.  The flow from the database into the CS works well.  No issues there.

    But, a search of the metaverse for the group shows an empty member attribute.  The sync process is not throwing any errors.  At least they're not showing up in the sync service app or the event logs.

    Where allowed, I'm using rules extensions for everything.  I can't use a rules extension to set the member attribute because it's an rdn.

    I'm going to move forward with this by extending the metaverse schema and adding a multi-valued string attribute named "memberOf" to the person object.  Then, I'll modify my existing MA to use that attribute instead of the member attribute.  I'm not sure what kind of issues I'm going to run into when exporting that to AD.  I'll cross that bridge when I come to it.  I don't anticipate that being an issue as the dns for all these objects will be calculated by the ADMA based on locations, group functions and person types (bascially, I don't care about the MV rdn).

    Anyway, I'm looking for some real world insight on this.  This whole effort is to migrate off an existing IDM system that works very, very well but quite expensive to license.

    Thanks,

    Greg Wilkerson

    Tuesday, March 25, 2014 2:39 PM

All replies

  • Note that memberOf calculated in AD, and not a real attribute so you can't flow out to this if that's your end goal. 

    A good way of troubleshooting things is to run previews on the CS objects, from there you can see what's happening to each attribute flow, and what will happen upon a full sync. If you preview a group, you'll probably see the member attribute have a status of "Skipped not precedent" or"Applied Delete" or something along those lines. 

    Do all of your users exist in the groups MA CS? The member attribute is a multivalued reference. To be able to reference the user it needs to be in the same CS and joined to an MV object IIRC. 

    Wednesday, March 26, 2014 12:29 AM
  • Cameron,

    I want to clarify the memberOf attribute so I understand what you're saying.  If, by "calculated" you mean the person.memberOf is calculated from the group.member attribute, I get that.  If this is calculated some other way, I'm confused.  The current IDM system manages groups by manipulating the group.member attribute.

    I've seen a few posts with this "Skipped not precedent" message.  I'm not sure where this message is supposed to appear.  But, nothing like that shows up in any sync status as viewed from the Synchronization Service app (at least no where I can find).

    And, the CS for the source MA is correctly populating the member attribute.  The rdns are added/removed based on what I do in the group table in the DB.  That works well.  Those member attributes are simply not flowing into the MV (the group.member attribute in the MV is empty). And, the group and person objects exist in the MV.

    I may have an anchor attribute issue.  In order to facilitate the renaming of group names, the anchor attribute for the CS is an id (mapped to uid).  My join rules for membership use the group name.  I have logging code written into the rules extension methods that indicate the sync is finding the correct object in the MV, though.  I'm going to change those rules to use the uid.  The uid does flow into the MV, so changing this would be easier and, honetly more straight forward.  I can't see where this would cause a problem, but who knows.

    Greg

    Wednesday, March 26, 2014 2:03 PM
  • memberOf isn't an attribute you can manage within Active Directory. So planning a solution where you control group memberships this way isn't viable. 

    If you search the connector space of your SQL Groups MA, find a group then click preview. From there you can see what's happening on the import flow and what is actually happening to the member attribute. 

    If they are in the CS but not in the MV this would indicate that either MV attribute precedence is incorrect, or you are unable to reference the users that exist in the CS. Having a read through this may give you a bit more of an idea about reference attributes. 

    Thursday, March 27, 2014 12:47 AM
  • Hi Cameron,

    There was never any intent to manage groups in AD using the memberOf attribute.  I'm good with that.

    The MV attribute precedence is something I'll have to look at.  I've not set any precedence on the attributes.

    That link is dead; comes back with a page that simply says "Bad Request".

    I can browse the groups in the CS and the members are set as expected.  No issues there. I've not tried to search the CS.  I'll do that tomorrow.

    Thanks,

    Greg

    Thursday, March 27, 2014 3:05 AM
  • Thursday, March 27, 2014 3:18 AM
  • Hi Cameron,

    That article was quite enlightening and vindicates and validates an approach I was considering.  The statement "all referencing and referenced objects must be in the same connector space" is my exact problem.  They don't.  Persons are process by one MA, groups (the self-owned attributes) by another.  It's not that I don't want them in the same connector space, but I don't want the group and person objects in the same MA. I don't care which MA process the memberships.

    I remember seeing an example setup somewhere on the net that had group and person objects within the same MA.  I ignored that because the implementation clutters up the data source the MA reads from.

    Later on in this article, Megan mentions using "dereferencing" attributes.  That's where I was going with the "memberOf" attribute I was going to create in the MV for the person object.  I can't use the member attribute as it's a referenced attribute.  That would provide some relief with regards to the referential integrity within the CS and MV.  The data source is guaranteed to be correct, so I don't need that in FIM.  I would then re-reference those member attributes during the AD export.

    So, thanks for the info.

    Back to work.  In an effort to be complete, I'll post something more once I get this working.

    Thursday, March 27, 2014 7:23 PM
  • First off, you already know how AD treats member and memberOf - this doesn't seem relevant here since you're just importing from SQL and interested in what the metaverse is doing.

    You haven't written out your flow rules so it's hard to really answer but it sounds like you're importing user objects and group objects from an SQL database and you're flowing the memberOf attribute into the group objects and expecting the member attribute to be magically populated for each user. This isn't going to work.

    If you want the member attribute for each user populated in the metaverse, one way to do that would be to create an advanced inbound attribute flow to the memberOf attribute of your users and in the extension code do a metaverse lookup looking for groups. Go through each group you find, check the member attribute and if the user is there, add that group to the users member attribute.

    There are some problems with this approach (what if the group isn't in the metaverse yet?) and also you're duplicating information: the information about which user is in which group already exists on the group objects so do you really need each user to have a member attribute in the metaverse? Depending on what you want to use it for you could leave it out of the metaverse and check for memberships on the outbound flow/s.


    Thursday, March 27, 2014 9:45 PM
  • Hey Dan,

    You're partially right on the member flow.  I think I chose poorly the name "memberOf" as my metaverse attribute, as it gets confused with the person.memberOf attrbute in AD.  They really will never be directly correlated or flowed to.  I'll use the group.member attribute.

    The MV does not contain a memberOf attribute for a person object out of the box.  I'm going to add it as a multi-valued string attribute. Then use the flow rules to populate it like any other multi-valued attribute.  At least that's my intent.  The article Cameron linked to above mentions this method.  It was nice to read that, as it validated my original thoughts about it.

    Because I'm using one MA for users and another MA for groups, the MV sync process won't link them.  I found that out today (thanks to Cameron).  I'm not sure I understand why that's not allowed, but that's cool.  Honestly, I'm not at all heartbroken that I won't be using the MV referential integrity.  The data, both users and groups, from the source is golden and the sole authority. 

    I'm using extension code for everything, and that assignment will be done in that code.  I won't be looking up the group in the MV.  I'm not concerned with the group being absent in the MV.  I plan on running the group import/sync before the user import/sync anyway.

    I've re-written and re-created MAs and the extension code numerous times.  I'm getting pretty good at it.

    Greg

    Thursday, March 27, 2014 10:45 PM
  • Well,  I have this working using the de-referenced attributes method.  It's pretty straight forward, as far as FIM stuff goes.  I created a new multi-valued string attribute in the metaverse called "memberOf" for the person object. Then, configured the flow to pull that value from the multi-table and stuff it into the CS. 

    The attribute reconciliation works well in the CS.  The attribute values follow theI had to write reconciliation code in the rules extension to handle the multiple values in the MV.  They show up as value collections.  It's no big deal, though.  I kind of expected that, but wasn't sure how FIM was going to deal with that.

    That was my major hang up with this.  I think I'm good now.

    Thanks Cameron and Dan.  I appreciate it.

    Friday, March 28, 2014 8:53 PM
  • Well, I thought I was good.  In this article (link copied from above)  http://social.technet.microsoft.com/wiki/contents/articles/331.understanding-reference-attributes-processing-in-fim.aspx there is a mention of "A self-owned reference attribute can exist in the metaverse and the synchronization engine is capable of transforming a self-owned reference attribute into a reference attribute during the metaverse to connector space transition". 

    How is that done? 

    I have tried several different variations of export flows from the MV into AD.  I'm beginning to think I'm going to have to create another MA (not sure which MA to use), push the person object and group object into that MA (the goal is to get them in the same CS) and set it there.  But, I don't see the difference in doing this and a newly created MA versus making the assignment directly in the AD MA.  Both objects exist in the AD MA.

    Thursday, May 15, 2014 6:02 PM
  • Are you trying to push out members of group or are you trying to export your multivalue string
    "memberOf" to another string attribute in AD? Perhaps you can provide some screenshots of your MV object and attribute flows and explain your desired goal.

    Friday, May 16, 2014 12:29 AM
  • Let's see.  To recap, the users are imported into the MV via a SQL MA.  The group names are imported into the MV using a different SQL MA.  I created a  multi-valued, string attribute named "memberOf" in the MV and flow the names of the groups into that attribute via the same SQL MA the user objects use.  I export everything to AD via an ADMA.

    My intent was to transform the user.memberOf string values to reference values the ADMA would export to AD.  That concept is mentioned in that document.  So, in a nutshell, that's what I want to get working.

    Below are screen shots.  This is a work in progress (I'm trying LOTS of different approaches).  Specifically, the MAs are configured for a direct mapping for the member attributes (a futile attempt to make this work).  Originally, the mapping in the user SQL MA was advanced, used the multi-valued string attribute memberOf and handled with extension code.

    I realize this may be hard to follow.  It's hard for me to explain.  If this is too confusing, I'll change the mappings back.

    This system is currently being managed with eDirectory drivers (Novell) and these reference value limitations simply do not exist with those.  I'm trying to get away from the eDirectory technology.


    Friday, May 16, 2014 2:44 AM
  • Friday, May 16, 2014 2:45 AM
  • Do you have access to the tables for these? Could you create a multivalue table to populate the member attribute of the group metaverse object, you can directly export these: http://technet.microsoft.com/en-us/library/cc708679(v=ws.10).aspx

    
    
    
    
    Friday, May 16, 2014 4:23 AM
  • Scrap everything and use the PowerShell MA for everything.
    Friday, May 16, 2014 4:25 AM
  • Hey Cameron,

    I have total control of all the DB tables FIM is accessing.  I build them up as part of IDM process.

    I've read this article, along the many others that address the "manager" scenario.  This really doesn't apply in this case as the user and group objects are loaded in separate MAs.  Getting reference values to flow with both object live in the same CS shouldn't be an issue. 

    I also saw a solution where the group and user objects were in the same table and differentiated by the "object_type" value (user, group).  That solution solved the issue of the groups and user being in the same CS.  As I grow tired of my daily FIM beatdown, that solution is growing more attractive.  That's a major DB redesign, and seems quite inefficient.

    The multi-value table for group memberships already exists in the DB.  For FIM purposes, I transferred that data into the user object multi-value table.  See screen shot.  I can certainly configure the group MA to access that multi-value table and load the group members as references.  But, because the group MA CS will not contain the user objects, I don't see how the references will be set.  If the reference value isn't set in the CS, it's not going to flow into the MV (at least I haven't figured out a way to set the an reference value for an object in the MV - my problem all along.

    This whole "setting a reference value" encompasses much more than just group memberships in my implementation.  Telephone resources and physical access (key cards, etc) are provisioned through the existing eDirectory system.  These objects exist in our current IDM system and are associated with users based on rules.  So, the reference value process is something I need to figure out, if I'm going to use this product.

    Maybe I could use a stripped down ECMA2 as a "staging" CS, export the users and groups into this CS and assign the reference values, then import the groups back into the MV, memberships intact.  I'm not sure that would get me where I want to go, and it seems like a lot of extra "stuff" to solve what should be a simple problem.  Hmmmmmm.  Or, connect the ECMA2 directly to my group membership multi-value table in the DB.  Hmmmmmm.  I'd still have to export the groups and users into that CS, but the import might be much more straight forward.  Hmmmmmm.

    The structure of my GroupMembership table (both columns are anchors or directly translatable to anchors):

    EmployeeGroups
    (
        GroupName varchar(50) not null,
        EmployeeID nvarchar(50) not null,
        ID int identity(1,1) not null
    )

    Friday, May 16, 2014 1:30 PM
  • Dan,

    I actually laughed out loud when I read this.  Funny.  Honestly, everything the system needs to operate lives in a single DB (lots of external interfaces).  If I chose that route, I'd bypass FIM all together.  However, I think doing all this in PowerShell would be unruly and difficult to support. 

    Friday, May 16, 2014 1:35 PM
  • Well, I thought I was good.  In this article (link copied from above)  http://social.technet.microsoft.com/wiki/contents/articles/331.understanding-reference-attributes-processing-in-fim.aspx there is a mention of "A self-owned reference attribute can exist in the metaverse and the synchronization engine is capable of transforming a self-owned reference attribute into a reference attribute during the metaverse to connector space transition". 

    How is that done? 

    I have tried several different variations of export flows from the MV into AD.  I'm beginning to think I'm going to have to create another MA (not sure which MA to use), push the person object and group object into that MA (the goal is to get them in the same CS) and set it there.  But, I don't see the difference in doing this and a newly created MA versus making the assignment directly in the AD MA.  Both objects exist in the AD MA.


    I take it by the deafening silence, that no one has a clue about the "dereferenced" reference attributes mentioned in this article?
    Friday, May 16, 2014 1:38 PM
  • GregW, did you ever get an answer on this? I have the exact same scenario and stumbled on the same "dereferenced" self-owned attribute paper. Using string self-owned attributes and transforming them to references automatically between MV and CS... sounded GREAT but I haven't found any docs on how to accomplish it. :)

    I have much the same need as you, separate SQL MA's for groups and person. Unless I want to load all 500k persons into the group table to create true references, I'm stuck. (The persons already exist in the MV from a different MA)

    -Ken

    Tuesday, February 23, 2016 8:56 PM
  • Ken,

    I think there is a massive disconnect between those documents and the real world stuff.  For the most part, if a document is over a few years old, it's useless.

    I do have this working though.  For some reason, the powers at FIM/MIM decided that to create a reference, it has to exist in the same connector.  So, I created a special MA that hooked into a SQL database pulling the bare minimum pertaining users and groups.

    I created a view that returns the PK for every user (employeeId) and every group (group name) in one column and the object type ("group" or "user).  Just two columns in the view.

    Then, I created a multi-valued attribute table containing the group PK, the word "member" (the actual attribute for the group object) and the PK of the user (employeeId).

    I've attached an image of some of this.  There's a bit to it, but it works well if you have the data structure in place.

    BHOLD will do this, too.  But it's a bit of work to set that up.

    Are you at Wash U. in St. Louis, or in Washington State.  Ask away.  I'll help any way I can.

    Greg

    Tuesday, February 23, 2016 9:39 PM