none
FIM Extensible Connectivity file Bassed MA not Consuming FIM output. RRS feed

  • General discussion

  • Good Morning All FIM Guru's

    I'm Reasonably new to FIM and trying to create my first Extensible Connectivity MA.

    Facts first - I am using FIM 2010 - We have not updated the FIM Extensible Connectivity Version 2 as I understand this needs FIM 2010 R2. I do not currently have the FIM portal running on my local dev environment also have not been able to get VS Debug to identify any issues. There are also no windows event logs which show any problem.

    I've followed through the Microsoft TechNet example successfully     ... technet.microsoft.com/en-us/library/cc720584(v=ws.10)

    I've now decided to try a real world example and am trying to consume a JPG file - small project with a lot of potential as outlined in Soren's excellent example

    blog.goverco.com/2011/12/loading-picture-to-ad-via-fim.html

    Soren's code works correctly - I have the output file created in

    C:\Program Files\Microsoft Forefront Identity Manager\2010\Synchronization Service\MaData\imageProject

    and have tried everything I can to get FIM to consume the output but at this stage have been un-successful.

    data looks like :

    id: 99536
    picture: /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJC....

    id: 99537
    picture: /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJC.... 

    +++++++++++++++

    Configuration; 

    Variables configured

    MaxWidth = 250

    SourceDirectory = c:/tmp/staff

    FileSearchFilter = *

    RegExReplaceFilter = .JPG

     

    Template File: format = AVP

    id:
    picture:
    <SPACE>

     Map Object Type = Person/Person

    No Filter

    Join on ID

    Also tried creating a new Metaverse Object with just id: and photo: attributes

    Mappings are all direct

    Have ensured that

     there is a Projection rule for the associated Metaverse object depending on which test I was running.

    and "Make Them Disconnectores" for deprovisioning

    Code from the dll as follows; - note that I've created a file in c:/tmp to confirm that the file is being created as expected. .... and it is.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using Microsoft.MetadirectoryServices;
    using System.Drawing;
    using System.Drawing.Imaging;
    using System.IO;
    using System.Text.RegularExpressions;
    using System.Runtime.InteropServices;
    
    namespace CardExchangeImage
    {
      public class Sample_CallBased_Extension : IMAExtensibleFileImport, IMAExtensibleFileExport
        {
    
          [DllImport("kernel32.dll", CharSet = CharSet.Auto)]
          public static extern void OutputDebugString(string message);
    
            // This allows us to resize the image. It prevents skewed images and 
            // also vertically long images caused by trying to maintain the aspect 
            // ratio on images who's height is larger than their width
            public static Image ResizeImage(Image FullsizeImage, int NewWidth, int MaxHeight, bool OnlyResizeIfWider)
            {
                // Prevent using images internal thumbnail
                FullsizeImage.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone);
                FullsizeImage.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone);
    
                if (OnlyResizeIfWider)
                {
                    if (FullsizeImage.Width <= NewWidth)
                    {
                        NewWidth = FullsizeImage.Width;
                    }
                }
    
                int NewHeight = FullsizeImage.Height * NewWidth / FullsizeImage.Width;
                if (NewHeight > MaxHeight)
                {
                    // Resize with height instead
                    NewWidth = FullsizeImage.Width * MaxHeight / FullsizeImage.Height;
                    NewHeight = MaxHeight;
                }
    
                System.Drawing.Image NewImage = FullsizeImage.GetThumbnailImage(NewWidth, NewHeight, null, IntPtr.Zero);
    
                // Clear handle to original file so that we can overwrite it if necessary
                FullsizeImage.Dispose();
    
                // return resized picture
                return NewImage;
            }
    
    
          public void GenerateImportFile(
            string fileName,
            string connectTo,
            string user,
            string password,
            ConfigParameterCollection configParameters,
            Boolean fFullImport,
            TypeDescriptionCollection types,
            ref string customData
            )
            {
    
    
    
    
    
              // TODO: Remove this throw statement if you implement this method.
    
                int NewWidth = Int32.Parse(configParameters["MaxWidth"].Value);
                string SourceDir = configParameters["SourceDirectory"].Value.ToUpper().Trim();
                string FileSearchFilter = configParameters["FileSearchFilter"].Value.ToUpper().Trim();
                string RegExReplaceFilter = configParameters["RegExReplaceFilter"].Value.ToUpper().Trim();
    
                string[] files = Directory.GetFiles(SourceDir, FileSearchFilter, SearchOption.TopDirectoryOnly);
    
                // StreamWriter f = new StreamWriter(fileName + ".tmp", false, Encoding.Unicode);
                //StreamWriter f = new StreamWriter(fileName, false, Encoding.Unicode);
                 StreamWriter swImport = new StreamWriter(fileName);
    
                //f.WriteLine("objectclass, delta, anchor-attribute, name, email");
                foreach (string file in files)
                {
     
    
                    System.IO.FileInfo fi = new FileInfo(file);
                    Image imageData = System.Drawing.Image.FromFile(fi.FullName);
    
                    Image resizedImage = ResizeImage(imageData, NewWidth, NewWidth, false);
    
                    string picture;
                    MemoryStream memory = new MemoryStream();
                    resizedImage.Save(memory, ImageFormat.Jpeg);
                    picture = System.Convert.ToBase64String(memory.ToArray());
                    memory.Close();
                    memory = null;
    
                    string Id = Regex.Replace(fi.Name.ToLower().Replace(string.Format("{0}", fi.Extension.ToLower()), ""), RegExReplaceFilter, "");
    
                    using (System.IO.StreamWriter file2 = new System.IO.StreamWriter(@"C:\tmp\log.txt", true))
                    {
                       
                        file2.WriteLine(string.Format("id: {0}", Id));
                        file2.WriteLine(string.Format("picture: {0}", picture));
                        file2.WriteLine();
                    } 
    
                  //  string Id = Regex.Replace(fi.Name.ToLower().Replace(string.Format("{0}", fi.Extension.ToLower()), ""), RegExReplaceFilter, "");
                    // try CSV FILE
                        
                   // f.WriteLine(string.Format("id: {0}", Id));
                   // f.WriteLine(string.Format("picture: {0}", picture));
                   // f.WriteLine();
                    swImport.WriteLine(string.Format("id:{0}", Id));
                    swImport.WriteLine(string.Format("picture:{0}", picture));
                    swImport.WriteLine();
    
                }
               // f.Close();
    
                // File.Copy(fileName + ".tmp", fileName, true);
    
                swImport.Close();
    
            } 
            
          public void DeliverExportFile(        
            string fileName,
            string connectTo,
            string user,
            string password,
            ConfigParameterCollection configParameters,
            TypeDescriptionCollection types
            )
            {
              // TODO: Remove this throw statement if you implement this method.
                throw new EntryPointNotImplementedException();
            }
        }
    }
    
    

    Not sure what the next step is - given that the Microsoft example writes directly to the output file I've also changed the code to do the same and still FIM does not consume the contents of the file...

    Suggestions from here ?

    Is there some way to identify why FIM will not consume the file ?

    I've tried to turn it into a CSV and create a completely new MA - not successful and given that there may be"," in the picture string a CSV is probably not the best option....

     Please if someone could test this and confirm what I've done wrong - I need to know as it's the knowledge I seek - the solution is secondary.

     Thanks

     

     

    Friday, October 26, 2012 2:29 AM

All replies

  • What exactly are you doing to get FIM to consume the file?  For an XMA, the file is generated by your code during an import operation and then imported by the sync engine to the connector space.  The synchronization step does your joins, projections, and attribute flows.  At what step are you failing?  Import or sync?  You can combine them in run profiles, but when starting out it is probably best to have separate steps.

    The application log and the operations log in Sync Manager can both provide (often the same) information as to what is happening to cause your MA run to fail, especially if errors are thrown.  If you see errors, post the details.  If not, more information about what you are actually _doing_ to get FIM to load the data would be helpful.

    Chris

    Friday, October 26, 2012 2:17 PM
  • Aside from Chris' notes, a few things come to mind:

    • Make sure the file is actually getting dropped to the MaData folder and that the file configured in the Full Import (or Delta Import) step matches the file you're dropping.
    • In lieu of P/Invoke'ing the OutputDebugString API, you can use the in-box Tracing classes in .Net
    • Make sure you're disposing your GDI resources (and that MemoryStream) when you do the photo manipulation. I've seen a few of these GAL Photo implementations and that's often an issue.
    • AD usually wants these images to be <=10KB in size, so you'll probably want some compression code as well


    My Book - Active Directory, 4th Edition
    My Blog - www.briandesmond.com

    Friday, October 26, 2012 7:02 PM
    Moderator
  • Thanks Chris,

     I was actually doing a full import and sync however after doing the Full import alone I am not seeing anything in the import results.

    adds 0
    updates 0

    the file does look the same -

    Pictures tell a 1000 words however apparently I need to have my account authorized....

    Thanks for the response though - there is a status of "Stopped-extension-dll-exception"

    which seems to point to something wrong with my dll...

    I did find the following error's in the Windows Application logs which points to memory issues...

     EventData

       System.OutOfMemoryException: Out of memory. at System.Drawing.Image.FromFile(String filename, Boolean useEmbeddedColorManagement) at CardExchangeImage.Sample_CallBased_Extension.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData) in C:\Users\skent\Documents\Visual Studio 2010\Projects\CardExchangeImage\CardExchangeImage\cardExchange.cs:line 90 Forefront Identity Manager 4.0.2592.0

    Thursday, November 1, 2012 3:47 AM
  • Thanks Brian, what do you mean by •Make sure you're disposing your GDI resources (and that MemoryStream) when you do the photo manipulation. I've seen a few of these GAL Photo implementations and that's often an issue. I expect your onto something here -
    Thursday, November 1, 2012 3:56 AM
  • Thanks Chris,

     I was actually doing a full import and sync however after doing the Full import alone I am not seeing anything in the import results.

    adds 0
    updates 0

    the file does look the same -

    Pictures tell a 1000 words however apparently I need to have my account authorized....

    Thanks for the response though - there is a status of "Stopped-extension-dll-exception"

    which seems to point to something wrong with my dll...

    I did find the following error's in the Windows Application logs which points to memory issues...

     EventData

       System.OutOfMemoryException: Out of memory. at System.Drawing.Image.FromFile(String filename, Boolean useEmbeddedColorManagement) at CardExchangeImage.Sample_CallBased_Extension.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData) in C:\Users\skent\Documents\Visual Studio 2010\Projects\CardExchangeImage\CardExchangeImage\cardExchange.cs:line 90 Forefront Identity Manager 4.0.2592.0


    The exception itself is likely a misnomer. There is an issue with the file you're trying to load. I'd get the image processing code working in a console app first and then bring it into FIM.


    My Book - Active Directory, 4th Edition
    My Blog - www.briandesmond.com

    Thursday, November 1, 2012 3:53 PM
    Moderator
  • Also, you need to update the build of FIM you're on. 2592 is the RTM version. I think 3627 or thereabouts is the current rollup for FIM 2010.

    My Book - Active Directory, 4th Edition
    My Blog - www.briandesmond.com

    Thursday, November 1, 2012 3:54 PM
    Moderator
  • Thanks Brian, what do you mean by •Make sure you're disposing your GDI resources (and that MemoryStream) when you do the photo manipulation. I've seen a few of these GAL Photo implementations and that's often an issue. I expect your onto something here -

    You're working with unmanaged resources. If you don't properly dispose them (e.g. via a using block), you're going to leak memory.

    My Book - Active Directory, 4th Edition
    My Blog - www.briandesmond.com

    Thursday, November 1, 2012 3:55 PM
    Moderator
  • I think that you have mistakenly commented out the line that actually creates the import file that FIM expects. You need, as Brian indicates, to write to the filename that FIM expects.

    // File.Copy(fileName + ".tmp", fileName, true);

    Regards, Soren Granfeldt
    blog is at http://blog.goverco.com | twitter at https://twitter.com/#!/MrGranfeldt

    Thursday, November 1, 2012 10:17 PM
  • I don't think it's a problem with the '.tmp' file as Stephen has also commented out other references to temp files, and the output is going directly to the fileName argument (as it really should, there is no need for temp files during MA processing).

    Agreed with Brian that all IDisposable resources, especially those having to do with GDI / Image / Drawing namespaces, should be wrapped in a 'using' block wherever possible.  This is also true for the StreamWriter.  I'd also suggest a try / catch around each attempt to load a file as an Image so the occasional non-image or rubbish graphic that will invariably appear can be logged and skipped over.

    Finally, I am not a big fan of GetThumbnailImage, as there are more flexible ways to resize a graphic that allow you to specify the quality, resampling method, etc.

    Friday, November 2, 2012 4:46 PM
  • Hi, Steve

    You're right. Looking at the code Again, I see that he's writting directly to the expected file. I usually end with a copy to prevent overwriting last import file if something goes wrong - but never mind.

    Is this an encoding problem? What encoding did you specify on the schema import? You need to provide a file in the same encoding when generating the import file. Also, not sure that it matters, but you have no Space after the colon (:) in the actual import file, but you have it in the log file. Not sure that matters though, but who knows - have never actually tested that come to think of it.


    Regards, Soren Granfeldt
    blog is at http://blog.goverco.com | twitter at https://twitter.com/#!/MrGranfeldt

    Friday, November 2, 2012 5:14 PM