locked
ECMA delta import - Determine if its Add or an Update? RRS feed

  • Question

  • Hi All,

    Seriously stuck here in implementing delta imports with my ECMA (ver 2.2, call-based), hoping someone here can guide me with the basic concepts around it.

    I'm writing a Lync provisioning ECMA, and my full import run profile works just fine. Now I need to move towards delta imports. In order to get changed objects, I use the DateTime of the last Import operation as the watermark and gather all objects which have changed since the last import operation. However, I'm stuck on how to detect which of those changed objects only has some changed attributes (i.e. they already exist in the connector space), and which objects are brand new. I can't find any documentation or examples on this :(. The code I'm using for my delta import is as follows:

    public GetImportEntriesResults GetImportEntries(GetImportEntriesRunStep importRunStep)
            {
                //System.Diagnostics.Debugger.Launch();
                GetImportEntriesResults importReturnInfo;
                List<CSEntryChange> csentries = new List<CSEntryChange>();

                // Create powershell environment
                InitialSessionState  initial = InitialSessionState.CreateDefault();
                initial.ImportPSModule(new string[] { "C:\\Program Files\\Common Files\\Microsoft Lync Server 2010\\Modules\\Lync\\Lync.psd1",
                                                      "C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\ActiveDirectory\\ActiveDirectory.psd1"});
                Runspace runspace = RunspaceFactory.CreateRunspace(initial);
                runspace.Open();
                PowerShell ps = PowerShell.Create();
                ps.Runspace = runspace;

    if (OperationType.Delta == m_importOperation) { ps.Commands.AddCommand("Get-csaduser"); ps.Commands.AddCommand("where-object"); ScriptBlock filter = ScriptBlock.Create("$_.Enabled -eq $True -and $_.SipAddress -ne $Null"); ps.AddParameter("FilterScript", filter); foreach (PSObject result in ps.Invoke()) { CSEntryChange csentry1 = CSEntryChange.Create(); csentry1.ObjectModificationType = ObjectModificationType.Update; csentry1.ObjectType = "user"; IList<CSEntryChange> existingcsentries = importRunStep.FullObjectEntries; // Returns 0 objects, why? PowerShell ps2 = PowerShell.Create(); ps2.Runspace = runspace; ps2.Commands.AddCommand("get-aduser"); string script = string.Format("(samaccountname -eq \"{0}\") -and (modified -gt \"{1}\")", result.Members["samaccountname"].Value,importRunStep.CustomData); ScriptBlock filter2 = ScriptBlock.Create(script); ps2.Commands.AddParameter("Filter", filter2); ps2.Commands.AddParameter("Properties", "*"); //ps2.Commands.AddCommand("select").AddArgument("employeeID"); foreach (PSObject result2 in ps2.Invoke()) { csentry1.AttributeChanges.Add(AttributeChange.CreateAttributeUpdate("FirstName", result2.Members["GivenName"].Value)); csentry1.AttributeChanges.Add(AttributeChange.CreateAttributeUpdate("LastName", result2.Members["Surname"].Value.ToString())); csentry1.AttributeChanges.Add(AttributeChange.CreateAttributeUpdate("EmployeeID", result2.Members["employeeID"].Value.ToString())); csentries.Add(csentry1); } //csentries.Add(csentry1); ps2.Stop(); ps2.Dispose(); } } importReturnInfo = new GetImportEntriesResults(); importReturnInfo.MoreToImport = false; importReturnInfo.CSEntries = csentries; importReturnInfo.CustomData = DateTime.Now.ToString(); ps.Stop(); ps.Dispose(); runspace.Close(); return importReturnInfo; }


    The code simply selects users from Lync who are still active and who have a SIP address and whose modified date is after the last import. The above code generally works if I modify a user (say change the firstname in AD), since I'm hardcoding the changetype as "update". However, if I provision a new user to Lync and run the Delta Import, the new user doesn't get added to the connector space (because I'm using Update and not Add)

    Now, how would I detect if the object changed already exists in the connector space? I can't find any way to tell if the modification type should be an "Add" or an "Update". I thought I could use the .FullObjectEntries property to retrieve a list of existing CSEntry objects in the connector space, but that returns 0 objects (even though there are 3 objects in the CS)

    Hope my question makes sense, and sorry for the shoddy code.

    Thanks in advance








    • Edited by kmittal82 Monday, June 17, 2013 10:52 PM
    Monday, June 17, 2013 9:56 PM

Answers

  • Unfortunately you can't retrieve a list of existing CSEntries in the connector space. You need to make the ObjectModificationType Add for all the changes you bring in. Once you pass it to the FIMSyncService, it will identity if it's add/update or delete. For a Full import you will need to run "Get-csaduser where{$_.Enabled -eq $True -and $_.SipAddress -ne $Null}" and pass all the objects in importReturnInfo. For a delta import check the last modified date in the same query and pass the subset of objects returned.  So basically the powershell query you run is different between a full import and a delta.

    Hope this helps. Have a play with it, you'll get the idea. 

    • Marked as answer by kmittal82 Tuesday, June 18, 2013 9:01 AM
    Tuesday, June 18, 2013 1:31 AM

All replies

  • Unfortunately you can't retrieve a list of existing CSEntries in the connector space. You need to make the ObjectModificationType Add for all the changes you bring in. Once you pass it to the FIMSyncService, it will identity if it's add/update or delete. For a Full import you will need to run "Get-csaduser where{$_.Enabled -eq $True -and $_.SipAddress -ne $Null}" and pass all the objects in importReturnInfo. For a delta import check the last modified date in the same query and pass the subset of objects returned.  So basically the powershell query you run is different between a full import and a delta.

    Hope this helps. Have a play with it, you'll get the idea. 

    • Marked as answer by kmittal82 Tuesday, June 18, 2013 9:01 AM
    Tuesday, June 18, 2013 1:31 AM
  • I am using separate queries for full import and delta imports using the same method you just mentioned (i.e. modification time stamp).

    I'll try putting everything as an Add, makes you wonder why the Update property is there :). Thanks a lot for taking the time to reply, looks like a bit of hit-and-trial is needed to understand ECMA. In the worse case scenario, I can get each users modification time as well as creation time, and check if creation time is newer than the last import then the operation should be an Add

    Once again, thanks for your help

    UPDATE

    I tried using Add instead of Update, and it works fine and FIM correctly detects object updates. However, if I manually add a new user to Lync and run a delta import on it, the new user is also shown as an "Update" instead of an "Add" in the statistics page. Not a deal breaker, just annoying. Thanks for all your help

    • Edited by kmittal82 Tuesday, June 18, 2013 1:06 PM
    Tuesday, June 18, 2013 9:01 AM
  • Delta import - update is kind of weird. That doesn't happen to me. What is the anchor do you use in the MA? I suggest having something like guid. 
    Tuesday, June 18, 2013 1:32 PM
  • Well I recreated my MAs and now updates do appear as updates and additions as adds - this is the second time my problem got fixed by recreating my MA, I'm now wondering whether I should delete it everytime I make a code change. The anchor I'm using is EmployeeID

    I tried it with both Add and Update (for ObjectModificationType), and it functions correctly. If I use Update, then no new users are detected when I do a delta import, but when I do an Add then both new users and updates are detected.

    The only issue now is that when I update a user and do a delta import, I do see the updated object but the Modification type is still shown as "Add", which also means that the columns for "Old Value" are all blank. Would have been nice to see which attributes got updated by seeing their Old values as well, but that column is always blank and only the New Values are shown. This behaviour is the same even if I set the ObjectModificationType to Update.

    Tuesday, June 18, 2013 3:09 PM
  • In reply to the above, I've figured out what I was doing wrong. I was doing a full import, changing the data and immediately did a delta import, no wonder why the Old Value column is empty! Need to do a sync BEFORE doing the delta import - once I did that the attribute changes were being reflected correctly in the "Old Value" and "New Value" columns.

    Safe to mark this thread as resolved, thanks for all your help

    Wednesday, June 19, 2013 11:10 AM