locked
select-string extremely slow RRS feed

  • Question

  • Hi

    I have a working script which I want to speed up a little.

    In this script I build two arrays of files. The first array contains invoices in xml format while the second array contains a list of the same invoices with the actions taken. Also in xml format.

    So I have a foreach loop to process each invoice, take out the referencenumber of each invoice and try to find that number in the second array with select-string. The second array is sorted bij lastwritetime and can contain over 1000 elements.

    My question is: The script does what I expect it to do but takes a long time processing each w. I suspect the select-string is consuming a lot of time. Is there a faster way to do this?

    My code: 

    foreach ( $w in $allw ) {
       [xml] $wx = Get-Content $w
       $refw = $wx.ref
       $wfound = Select-String -Pattern $refw -Path $processedw -List -SimpleMatch
       if ( $wfound ) { ...etc }


    Friday, May 17, 2019 1:49 PM

Answers

  • This all depends on the XPath you are using.  If the XPath is bad then the whole file will be scanned.  Without an XPath example there is no way to know your issue.  Also large XML files will load slowly and, in a loop, this may be very slow.


    \_(ツ)_/

    Friday, May 17, 2019 6:50 PM

All replies

  • I feel like you're sort of being a bit too general for us to get a real feel for what you're doing...

    Putting stuff in an array can be problematic, for example when reading lines from a file...  You typically have to scan start to finish.

    Friday, May 17, 2019 3:53 PM
  • This all depends on the XPath you are using.  If the XPath is bad then the whole file will be scanned.  Without an XPath example there is no way to know your issue.  Also large XML files will load slowly and, in a loop, this may be very slow.


    \_(ツ)_/

    Friday, May 17, 2019 6:50 PM
  • Hi,
    Was your issue resolved? 
    If you resolved it using our solution, please "mark it as answer" to help other community members find the helpful reply quickly.
    If you resolve it using your own solution, please share your experience and solution here. It will be very beneficial for other community members who have similar questions.
    If no, please reply and tell us the current situation in order to provide further help.
    Best Regards,
    Lee

    Just do it.

    Monday, May 20, 2019 5:29 AM
  • Hi,

    Was your issue resolved?

    If you resolved it using our solution, please "mark it as answer" to help other community members find the helpful reply quickly.

    If you resolve it using your own solution, please share your experience and solution here. It will be very beneficial for other community members who have similar questions.

    If no, please reply and tell us the current situation in order to provide further help.

    Best Regards,

    Lee


    Just do it.

    Monday, June 3, 2019 7:43 AM