locked
using -match inside foreach loops RRS feed

  • Question

  • O.k. so I've got a bit of a weird one here, first off I want to be clear that I have an alternate solution that works for my script, I'm just hoping someone can explain to me what's going on, because I sure can't figure it out, and I find I learn the most when PowerShell does something funky that seems like it should work, but just doesn't.

    I was working on a script that steps through one array, and then looks up values in a second array based on common data in the current entry of the first array, then updates the appropriate entry in the second array with a new field containing data from the current entry in the first array.

    To do this I was using the -match command like so:

    $Array1 -match "KnownValue" | Add-Member NoteProperty -Name PropertyName -Value MyValue

    This works beautifully if ran as a one-liner, it updates the one entry in $Array1 which matches the "KnownValue" with a new property with the appropriate name and value.

    So I move on to testing it by setting a variable to one of the entries in $Array1 and doing the match there as I don't have a value that's unique between my two arrays, I have 3 fields that in tandem are unique and I need to find the one entry in $Array1 that matches them.

    $Test = $Array1[5]
    $Array1 -match $Test | Add-Member NoteProperty -Name PropertyName -Value MyValue

    This also works correctly, wonderful.  So I move on to putting it all in a foreach loop to step through the other array and update my values. Here's what it all looked like after it came together.

    $Array1 = import-csv "MyCSVPath"
    $Array2 = import-csv "MyOtherCSVPath"
    foreach ($Item in $Array2){
         $Temp = $null
         $Temp = $Array1 -match $Item.Property
         # A bunch more code that makes sure I get a unique entry, trust me I verified that $Temp only ever has one entry in it.
         $Array1 -match $Temp | Add-Member NoteProperty -Name PropertyName -Value $Item.DesiredValue
    }
    $Array1 | export-csv "OutputFile.csv" -NoTypeInformation


    Imagine my surprise when my output file has the new property, but all the entries values are the same.  I went back and added in a write-host at the end of the loop and changed the Add-Member in the pipe to a write-host to output what the pipline is getting like this.

    $Array1 -match $Temp | %{write-host "$_"}

    I find that the match is passing through the entirety of $Array1 to the next step of the pipeline, not the single entry it's supposed to. 

    I figure I must have misunderstood how this works, so I run it outside the script as a one-liner with the write-host in the pipeline.  Only the one correct entry is written out to the console. 

    O.k. I must have messed something up with the $Temp variable, lets add in a write-host to the foreach loop to see what it has with every iteration.  It's working exactly as expected, it only has the one entry from $Array1, and it matches the data from the item for that entry of $Array2.

    Well crap, what do I do now.  It works as expected outside the foreach loop, but as soon as it's in the foreach loop the match seems to fail for some reason.

    I ended up solving the issue by replacing the match in the add-member line at the end with a where-object statement in the pipe like this:

    $Array1 | where{$_.UniqueProperty -eq $Temp.UniqueProperty} | Add-Member NoteProperty -Name PropertyName -Value $Item.DesiredValue

    So I've got my script up and running, but I'm hoping someone can explain to me why the original solution didn't work though so I can hopefully avoid banging my head against the wall for hours in the future due to me not understanding something, or a bug in the system.

    Sorry for the huge post, but thanks if you made it this far :)


    • Edited by Khelbun Thursday, March 21, 2019 12:52 PM Added code blocks to make this actually readable, sorry for missing that the first time.
    Saturday, March 16, 2019 12:42 AM

All replies

  • I cannot exactly tell you why -match behaves like it does I just can tell you that you have some missconception about it. The -match operator does a comparison using regular expressions. It is made to be used for string comparisons. If you use it for array operations you will get unpredictable results.

    An example: let's have an array of 20 common names and use the match operator to try to pick just one particular name from it .... like this:   (before you run the code try to predict the result please!  ;-)  )

    $NameToCheckFor = 'Liam'
    $Arra1 = @(
        "Liam",
        "Noah",
        "William",
        "James",
        "Logan",
        "Benjamin",
        "Uilliam",
        "Elijah",
        "Oliver",
        "Jacob",
        "Wyliam",
        "Olivia",
        "Ava",
        "Isabella",
        "Wiliam",
        "Mia",
        "Charlotte",
        "Amelia",
        "Viliam",
        "Abigail"
        )
    $Arra1 -match $NameToCheckFor 

    This comparison outputs every single element of the array the -match operator finds a match for.

    The behaviour you're looking for you'll only get with the appropriate operators like -in, -notin, -contains and -notcontains. To learn more about you should read the (complete) Get-Help about_Comparison_Operators.

    If you have an array of object with more than on property per object it gets even more complex. This way you need to specify the exact property you want compare to get a predictable result.

    So if we use the appropriate operator for our comparison example we get the expected result:

    $NameToCheckFor = 'Liam'
    $Arra1 = @(
        "Liam",
        "Noah",
        "William",
        "James",
        "Logan",
        "Benjamin",
        "Uilliam",
        "Elijah",
        "Oliver",
        "Jacob",
        "Wyliam",
        "Olivia",
        "Ava",
        "Isabella",
        "Wiliam",
        "Mia",
        "Charlotte",
        "Amelia",
        "Viliam",
        "Abigail"
        )
    $Arra1 -contains $NameToCheckFor 

    I hope that helps you to understand.  ;-)

    Have fun!


    Live long and prosper!

    (79,108,97,102|%{[char]$_})-join''


    • Edited by BOfH-666 Saturday, March 16, 2019 1:27 PM
    Saturday, March 16, 2019 1:25 PM
  • Thanks for the reply, the reason that I'm using -match and not -contains or -in is because those only return a  $true or $false value to tell me if the object is in the array.  I'm looking for the actual entry of the object to be returned so that it can then be added to a different variable, or to update the returned object in the original array, which -match does.

    The note I've got in the middle of the posted code basically represents a bunch more checks using the other operators to ensure that I've got the correct entry based on multiple other fields if I get more than one entry returned.

    • Edited by Khelbun Thursday, March 21, 2019 12:56 PM Added 2nd paragraph to clarify additional checks being done.
    Thursday, March 21, 2019 12:47 PM
  • OK, but as I showed above that has some disadvantages. If you are awary of that ... I'm ok with it. ;-)

    Live long and prosper!

    (79,108,97,102|%{[char]$_})-join''

    Thursday, March 21, 2019 12:56 PM
  • Both match and contains return Booleans.  Arrays are tested with "contains" or "in" and strings are tested with "match".  "Match" with an array will not work correctly.


    \_(ツ)_/

    Thursday, March 21, 2019 1:06 PM
  • match with an array returns the object or objects that match the criteria, not a boolean.  It only returns a boolean when performed on a string.
    • Edited by Khelbun Thursday, March 21, 2019 3:54 PM Fixing my dyslexia, seriously though screw the guy who came up with the word dyslexia
    Thursday, March 21, 2019 3:39 PM
  • Yep, totally aware of the fuzzy logic issue with using match, and that it uses regex on basically all the values of every object contained in the array.  In this case it's actually what I need since I don't have a common unique value between the arrays (spelling errors, using aliases instead of the primary SMTP address, etc. in one of the datasets I'm pulling from.), and I need it to return the matching object instead of a boolean.

    I'm just trying to figure out if it's something I'm missing/misunderstanding, or a glitch in how it's supposed to work that's causing -match to pass the entire array over to the next step of the pipeline when in a foreach loop, but not out of it.

    Appreciate the help so far :)

    Thursday, March 21, 2019 3:51 PM
  • match with an array returns the object or objects that match the criteria, not a boolean.  It only returns a boolean when performed on a string.

    As long as you understand how to use that as a Boolean and wha tit means.

    if($a -match 2){

    This will be true if the item is found or any part of the item is found.  "Contains" will be true only if any item exactly matches the test.

    $a = 'abc','def','ghi'
    $a -match 'e'

    This will always match 'def'.

    $a -contains 'e'

    This will never match anything.

    $a -contains 'def'

    this will be true because it matches a complete entry exactly.


    \_(ツ)_/

    Thursday, March 21, 2019 5:37 PM
  • O.k. I feel like we're getting off topic a bit here, but there seems to be a bit of confusion in how -match operates in the way that I'm using it that's clouding the issue.

    Here's the documentation on comparison operators that was helpfully linked above, I've bolded the important bit:

    "

    -match

    Description: Matches a string using regular expressions. When the input is scalar, it populates the $Matches automatic variable.

    The match operators search only in strings. They cannot search in arrays of integers or other objects.

    If the input is a collection, the -match and -notmatch operators return the matching members of that collection, but the operator does not populate the $Matches variable.

    "

    This means that when you perform a -match against an array it will search thorough all strings in every property of every object in the array, and then return any objects in the Array that match the comparison criteria. 

    It does not return a boolean value when being used on an array.

    For instance, in your first example above:

    $a = 'abc','def','ghi'
    $a -match 'e'

    It would return the string 'def'

    If you were to have this in your array instead:

    $a = 'abc','cde','efg'
    $a -match 'e'

    It would instead return an array containing two string objects 'cde' and 'efg'

    This behaviour is especially useful when dealing with large arrays of objects that have multiple string properties such as arrays of AD user objects that you want to search for specific values and return the user object that you're looking for.

    The question here is why is it returning the entire array if, and only if, it is being used with the pipeline inside of a foreach loop.



    • Edited by Khelbun Thursday, March 21, 2019 10:30 PM Apparently I suck at formatting on this site, why oh why can't forum software show you what will actually be posted in the editor.
    Thursday, March 21, 2019 10:27 PM
  • It returns a match or nothing which is the same as a Boolean.  Any value is true and any null is always false.

    While the document is correct it does not address how we can use the behavior.


    \_(ツ)_/

    Thursday, March 21, 2019 11:03 PM
  • Sure in many situations in PowerShell you can use the fact that there's an object or not similarly to a boolean $true/$false check, they aren't the same and that doesn't make it a boolean return, but you can treat them the same in many cases. 

    I'm not sure what that has to do with the topic of the thread, or what we've been talking about though as I'm not using the return of the -match command for a boolean check, I'm using the Object that it returns to do work.  I'm looking up a specific object in array one based of off string matching criteria from properties in the current object in array two, then updating the object returned from array one by the match operator with a property from array two.

    Friday, March 22, 2019 6:49 PM
  • Then you need to be more explicit about your issue.  Either "match" works the way you want or it doesn't.  What about your code is not working?  Note that "match" can return a collection.

    \_(ツ)_/

    Friday, March 22, 2019 7:30 PM
  • If I use -match on my array with the comparison being a variable set to an object pulled from the array and pass it to the pipeline to add a value like so:

    $Test = $Array1[5]
    $Array1 -match $Test | Add-Member NoteProperty -Name PropertyName -Value MyValue

    Everything works as I want, the object in $Array1 that matches my comparison $test gets passed to the pipeline and updated with a new NoteProperty set to the value I want, and none of the other objects in $Array1 are touched.

    If I do the exact same thing inside a foreach loop like so:

    $Array1 = import-csv "MyCSVPath"
    $Array2 = import-csv "MyOtherCSVPath"
    foreach ($Item in $Array2){
         $Temp = $null
         $Temp = $Array1 -match $Item.Property
         # A bunch more code that makes sure I get a unique entry, trust me I verified that $Temp only ever has one entry in it.
         $Array1 -match $Temp | Add-Member NoteProperty -Name PropertyName -Value $Item.DesiredValue
    }

    Every item in $Array1 gets passed to the pipeline and gets updated with the current value for every pass of the foreach loop.  This is undesirable as it results in the new Property added to $Array1 being set to the value from the last pass of the foreach loop for every object in $Array1, and the whole point of this is to update each object in $Array1 with a unique value from the corresponding entry in $Array2.

    I've verified that inside the foreach loop $Temp is getting set to a single object from $Array1 just like in the test code outside the foreach loop, so the only conclusion I can come up with is that -match is passing the entire source array to the pipeline, but for some reason only inside the foreach loop, and I'd like to know why.

    Saturday, March 23, 2019 12:04 AM
  • Does single object mean an object type or a ValueType?

    You area also using CSV which is NOT an array of items.  It is an array of "PsCuetomObjec". 


    \_(ツ)_/

    Saturday, March 23, 2019 12:10 AM
  • What you are getting back from a CSV o not a value but is an object.

    PS D:\scripts> $csv = @'
    >> Name
    >> joow
    >> mary
    >> sue
    >> '@ | ConvertFrom-csv
    PS D:\scripts> $csv -match 'mary'
    
    Name
    ----
    mary
    
    
    PS D:\scripts>
    


    \_(ツ)_/

    Saturday, March 23, 2019 12:13 AM
  • Yes both arrays are arrays of PsCustomObjects with multiple note properties set per object.

    One of the CSV files is a CSV export from PowerShell of AD users with common values such as name, email address, samaccountname etc.  The second is a manual list generated from an HR person with a bunch of User's names email addresses, and Employee IDs.

    So each array is made up of user objects with noteproperties with the users names, emails, etc. in them.

    When I say I verified that I'm getting a single object set to $temp I mean that only one user object from the source array is being set as the value. 

    When I then do a match against $temp to a pipeline as a one liner I'm getting the single user passed to the next step of the pipe.  When I do a match against $temp inside of a foreach loop I'm getting every user object from the source array being passed to the next step of the pipe.

    Saturday, March 23, 2019 12:24 AM
  • There is your problem.  The issue has nothing to do with loops.


    \_(ツ)_/

    Saturday, March 23, 2019 1:25 AM
  • The issue only occurs inside a foreach loop though.

    Using match against the array of objects with one of the objects as the criteria passes only the desired object to the pipeline when used normally, it only passed the whole array when used inside a foreach loop.

    Saturday, March 23, 2019 5:22 PM
  • You will have to provide an example of that shows that.

    I suspect that the access to the object in the match variable is not giving you what you think.  The object properties being matched would determine what is detected.  Matching a custom object can work differently when selecting a single item and when passing a pointer to an item or passing a copy of an item.  THat behavior is determined, in part, by the type of objects being asccessed and how they are being accessed.  "Match" works on simple arrays.  It match strings.  THe value of an object in an array when converted to a string will be different.  THe value bing match will likely be this for a CSV:

    Notice what happens in the following test:

    PS D:\scripts> $csv = @'
    >>  Name
    >>  joow
    >>  mary
    >>  sue
    >> '@ | ConvertFrom-csv
    PS D:\scripts> $csv[0].ToString()
    
    PS D:\scripts>
    

    It does not convert to anything.  Also the "match" takes a RegEx expression.  Characters in any string in a PsCustomObject will misbehave in a match.

    So the answer is tightly bound to the nature of the objects being referenced.  A little more testing and you will likely find the cause and, possible, a fix.


    \_(ツ)_/

    Saturday, March 23, 2019 6:57 PM
  • I appreciate the attempt to help, but I don't feel you actually read the original post.  In it I have example code, walk through the process I took, the behaviour both in and outside of the foreach loop and how I tested for it, my question as to why it's behaving the way it is, and lastly the fix I used to get my code working.

    This post was never about trying to find a fix but an attempt to get some clarity on why things are happening the way they are.

    Again I appreciate the attempt to help, but I don't think I'm going to get what I'm looking for here, as I've now spent several days trying to explain everything that was in the original post, and we're still not 100% there.

    Sunday, March 24, 2019 8:51 AM
  • I have run your code and cannot reproduce your issue no matter what I have tried.  Without your exact CSV files this would be impossible to do unless I can guess the contents of the files.

    I will give up now.  Maybe someone is better at guessing what you are doing then I am.


    \_(ツ)_/

    Sunday, March 24, 2019 8:57 AM