locked
how do i remove elements from this object (array?) RRS feed

  • Question

  • i am using following to read a file with 40000 line:

    # Read working file into an array
    # this file has ~ 40,000 employeeid - courseID pairs whith many-to many relationships
    $Array=import-csv -Path $inputFile -header "NID","courseID" 
    $Array=$kursimArray | sort NID

    I now need, based on var criteria remove some "lines" from this "array". for example i will check if NID equals an existing employeeid AD value and if NOT then i want to remove the "line" from the "array"

    this process will give me a much smaller "array" to cont. my manipulations

    i have been readin a ton about arrays and all and am getting confused by the moment.. can anyone pls show me the code to remove a "line".

    pls note the "many-to-many" relationships so i think a hashtable is out?

    tnx in advance


    roys99


    • Edited by roys99 Wednesday, November 7, 2012 4:10 PM
    Wednesday, November 7, 2012 4:08 PM

Answers

  • Try this:

    $str_array = get-content $inputfile
    foreach ($line in $str_Array) 
     {
      $str_NID  = ($line -split ',')[0]
      if (-not $HashTable[$str_NID]){  
          $Array = $Array -notmatch "^$str_NID"
       }
     }
     $array


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    • Marked as answer by roys99 Thursday, November 8, 2012 2:07 PM
    Thursday, November 8, 2012 12:02 PM

All replies

  • $array = @(
    '"John","Course1"',
    '"John","Course2"',
    '"Joan","Course1"',
    '"Barney","Course4"',
    '"Eric","Course5"',
    '"Barney","Course1"'
    )
    $array -notmatch '^"Barney"'


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    • Proposed as answer by Boe ProxMVP Thursday, November 8, 2012 1:10 AM
    Wednesday, November 7, 2012 4:16 PM

  • Roys99:
    Please check if the below helps by adding the required employee id to  $result
    #c:/test/roys99.txt
    #NID,CID
    #100,200
    #101,203
    #102,343
    #78,233
    #100,102
    #80,121

    #Code block begin
    #your employee id  list add them to the array accordinly
    $empArray = 70,100
    #empty array to hold result
    $result = @()
    foreach($eid in $empArray)
    {  
        $file = import-csv C:\test\roys99.TXT | Where-Object {$_.NID -eq $eid}    
        $result += $file  
    }
    #Code block end
    • Edited by Venkat786 Wednesday, November 7, 2012 8:18 PM
    Wednesday, November 7, 2012 6:30 PM
  • hi

    I tried it with my code and it does not work.

    foreach ($line in $Array) 
     {
      $cnt++
      $NID             = $($line.nid)
      $courseID        = $($line.courseID)
      $adObjName=$HashTable[$nid]  # VERY FAST ..
      if ($adObjName -eq $null)    # if no such user then we remove record
       {
        $cnt                       # count removals
        $Array -notmatch '^$NID'
       }
     }

    when the line $Array -notmatch '^$NID' is marked out all runs well and indeed I have many i need to remove but when line is "on" i simply get a dump to screen of entire array (40,000 lines takes a while) and none are removed.

    can u help me resolve this?


    roys99

    Thursday, November 8, 2012 7:14 AM
  • as i have 40000 lines will this not be very very slow?


    roys99

    Thursday, November 8, 2012 9:35 AM
  • Can you post the first 2 lines of your csv file?

    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Thursday, November 8, 2012 11:20 AM
  • 02352292,21098
    30523752,64610
    02131003,21098

    thank you

    roys99

    Thursday, November 8, 2012 11:52 AM
  • Try this:

    $str_array = get-content $inputfile
    foreach ($line in $str_Array) 
     {
      $str_NID  = ($line -split ',')[0]
      if (-not $HashTable[$str_NID]){  
          $Array = $Array -notmatch "^$str_NID"
       }
     }
     $array


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    • Marked as answer by roys99 Thursday, November 8, 2012 2:07 PM
    Thursday, November 8, 2012 12:02 PM
  • That will use a lot of memory, but will be faster if it works.  I'd use:

    [io.file]::ReadAllLines($file)

    ...because most of the time in your script will be reading the file into memory.


    Grant Ward, a.k.a. Bigteddy

    Thursday, November 8, 2012 12:17 PM
  • $Array=import-csv -Path $inputFile -header "NID","courseID" 
    $Array=$Array | sort NID
    

    the above only takes < 2 secs , reading the file is extremely fast, perhaps an ordered 2 col file is read all at once as it is only 623 kbs even with 40000 lines.

    but I will keep this info as it surely will be usefull.


    roys99

    Thursday, November 8, 2012 1:07 PM
  • as soon as i get back to office - tnx

    roys99

    Thursday, November 8, 2012 1:07 PM
  • 40K * 14 byte lines is a little over half a meg......

    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Thursday, November 8, 2012 1:11 PM
  • I gain about a second in read time on my laptop with a test file of 40,000 14 byte records switching to the raw dotnet method. 

     

    Not worth the obfucation factor, IMHO.


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Thursday, November 8, 2012 1:35 PM
  • I gain about a second in read time on my laptop with a test file of 40,000 14 byte records switching to the raw dotnet method. 

     

    Not worth the obfucation factor, IMHO.


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "


    I agree.  I did't think we were talking about such small amounts of data.

    Grant Ward, a.k.a. Bigteddy

    Thursday, November 8, 2012 1:38 PM
  • hi

    it works! thank you very much

    can u pls explain the -notmatch"^$str_NID" code  ?

    what is it called? the ^ and where should i direct my reading (key words)?

    thank you in advance..

    Roy


    roys99

    Thursday, November 8, 2012 2:09 PM
  • That is a regular expression.

    You can start with:

    get-help about_regular_expressions

    It's kind of a Swiss Army Pandora's Box.


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Thursday, November 8, 2012 2:51 PM