locked
PowerShell Runspace Slowness RRS feed

  • Question

  • A little background information.  I am managing two Azure AD tenants.  I am using PowerShell to create an Azure B2B sync between the two tenants.  Since I have to connect to two tenants simultaneously, I am using two runspaces to try and keep everything separate (this could also let me run commands in parallel to the two tenants.)

    My issue is, the longer the runspaces are active and the more commands they receive, the slower they get, almost to the point of being unresponsive.

    I have been able to verify this behavior using the following code.

    $Runspace = [PowerShell]::Create()
    
    $timer = New-Object -TypeName System.Diagnostics.StopWatch
    for($n=0; $n -lt 1000; $n++){
    $timer.Reset()
    $timer.Start()
    $Runspace.AddScript({
        Get-Date | OUt-Null
    }) | Out-Null
    $Runspace.Invoke()
    $timer.Stop()
    $timer.ElapsedMilliseconds
    }

    With this code, you will notice by the 1000th iteration a simple call to Get-Date will have slowed down significantly.  If you run a similar test without the runspace, nothing ever slows down.  My point with this test is to show, it's not the cmd-lets that are ran, but the runspace its self.

    I have a few questions. First, why do runspaces get progressively worse the longer you use them?  Are they more designed for simple queries with quick tear downs?  Is there a way to counterattack this behavior aside from recreating the runspace? Is this the best method to use for my scenario?  Maybe there is an easier way to keep two Azure AD module connections open simultaneously without using runspaces?

    Thanks in advance to anyone that can shed some light on this issue.

    Saturday, October 7, 2017 2:39 PM

All replies

  • You are causing a handle and memory leak.  Try it like this to get an accurate measurement without the leaks.

    $Runspace = [PowerShell]::Create()
    $Runspace.AddScript({
    		Get-Date | OUt-Null
    	}) | Out-Null
    
    Measure-Command {for ($n = 0; $n -lt 1000; $n++) {$Runspace.Invoke()}}
    


    \_(ツ)_/

    Saturday, October 7, 2017 5:01 PM
  • Thanks.  My next question is, how would you use the above script in an actual foreach loop.  For example, iterating through a list of users.

    • Edited by AndyHJ Monday, October 9, 2017 12:21 PM grammatical
    Sunday, October 8, 2017 6:40 PM
  • I found the issue and the proper way to work around it.

    For anyone that finds this thread.  After your invoke, you clear your commands

    $Runspace.Commands.Clear()

    This doesn't totally solve my problem, but I do have a better  understanding of Runspaces.

    Tuesday, October 10, 2017 6:01 PM
  • That clears the commands you have added.  You are not adding commands you are running a script that you have added. You can only add one script.  Adding it over and over should have no effect.  I can run that script forever with no slowness but ad in the timer and the slowness starts.

    Also adding the same script is silly as it only needs to be added once.  A script can take parameters and they can be different on each "Invoke".

    Notice that, in my code example, I only call the "Invoke" in the loop.  The script is added once before the invoke.

    You can stack many runspaces up with different scripts and choose them as needed. 

    This is all similar to using System.Threading.Tasks which also adds numerous other capabilities such as evented queuing where the thread is idle until you add to its input queue where it will process items until the queue is empty.

    With runspaces we can use the state changed event to create a pipeline of runspaces where one ends it triggers a second runspace that does further processing.

    I suggest that workflows are a better choice for most things that are complex because they provide semantics that allow us to easily state what needs to be don and allows the workflow system to implement the requests without the scripter needing to mess with the nuts and bolts of threading or managing runspaces.

    If you want to run tasks on multiple computers then the workflow used with "-AsJob" is a very powerful mechanism for implementing this in the background.  The job contains the output until we are ready to receive it.


    \_(ツ)_/

    Tuesday, October 10, 2017 6:27 PM
  • The results you are receiving are much different than mine.  

    First, your recommendation regarding the AddParameters() method makes a lot of sense.  I had been adding the script and the parameters in the loop.  Just changing the parameters does make a lot of sense.  Thank you.  However, if you want to change the script, I would still argue that clearing the commands is beneficial, see my results below.

    Second, regarding the timer and differing results. I ran the same script without the timer and I always receive the same slow response time, I can't measure this exactly as I don't have a timer setup.  So, I moved the timer outside the for-loop.  I think we can both agree, this shouldn't cause a slow down or memory leak of any kind. Hear is the command and the results. 

    $Runspace = [PowerShell]::Create()
    $timer = New-Object -TypeName System.Diagnostics.StopWatch
    $timer.Start()
    for($n=0; $n -lt 1000; $n++){
    $Runspace.AddScript({
        Get-Date | OUt-Null
    }) | Out-Null
    $Runspace.Invoke()
    }
    $timer.Stop()
    $timer.ElapsedMilliseconds
    105044

    The end result there is the output in milliseconds from the timer object.  In addition to this, if I run the following code, you'll notice there are a 1000 commands in the $runspace object.  Which is the exact number of times that the for-loop ran.

    $runspace.Commands.Commands | Measure
    
    
    Count    : 1000
    Average  :
    Sum      :
    Maximum  :
    Minimum  :
    Property :

    I noticed after adding multiple scripts to a runspace and running the Invoke command multiple times, I would receive some of my previous results along with the results from the new script.  I think the real cause of the slow-down is that each iteration of the for-loop is running the command/scripts n-times. 

    If we clear the commands after every loop, we should see drastic reduction in processing time.

    $Runspace = [PowerShell]::Create()
    $timer = New-Object -TypeName System.Diagnostics.StopWatch
    $timer.Start()
    for($n=0; $n -lt 1000; $n++){
    $Runspace.AddScript({
        Get-Date | OUt-Null
    }) | Out-Null
    $Runspace.Invoke()
    $Runspace.Commands.Clear()
    }
    $timer.Stop()
    $timer.ElapsedMilliseconds
    2254

    Which we do.  Down to 2254 ms from 105044 ms.  The other option would be to simply move the loop into the AddScript method.  This way the loop is ran inside the script.  However, clearing the commands is still necessary if you plan to run subsequent scripts using the same runspace.

    Thank you for the advice regarding workflows.  I am planning to look into them.  To be honest, I don't think runspaces will work for what I am trying to do anyway.  I manage multiple AzureAD tenants and I would like to be able to connect to multiple AzureAd tenants simultaneously.  I was hoping that runspaces would give me an isolated environment to connect to each Tenant individually.



    • Edited by AndyHJ Wednesday, October 11, 2017 2:25 PM
    Wednesday, October 11, 2017 2:22 PM
  • You keep reading the same script inside your loop.  Don't do that.

    $Runspace = [PowerShell]::Create()
    $Runspace.AddScript({
    		Get-ChildItem $args[0]| OUt-Null
    	}) | Out-Null
    
    $timer = New-Object -TypeName System.Diagnostics.StopWatch
    $timer.Start()
    for ($n = 0; $n -lt 1000; $n++) {
    	$Runspace.Invoke('c:\')
    }
    $timer.Stop()
    $timer.ElapsedMilliseconds

    This is the most efficient way to run a script in a loop.   Notice how we pass arguments to a runspace.


    \_(ツ)_/

    Wednesday, October 11, 2017 4:25 PM
  • I understand that.  I was just demonstrating that I isolated the slowness issue to the number of scripts being added.  Specifically, the addScript() method doesn't overwrite your last script you added, but they are added to the commands list.

    You are absolutely right though, the script should be added once and ran subsequently by adding parameters and running the Invoke() method.

    Thanks for the clarification.

    Wednesday, October 11, 2017 8:15 PM
  • I still don't understand why you would want to add the same command over and over again.

    \_(ツ)_/

    Wednesday, October 11, 2017 11:58 PM
  • I am not saying you would and I was definitely handling my initial script poorly. 

    However, there are instances where you may want to add different scripts.  The point being, adding another script to a runspace doesn't appear to override the original script.

    Thursday, October 12, 2017 12:54 PM
  • Well the last added script will  be the default.  Adding scripts is intended to be used to add variables and functions which then become compiled into the  runspace.  If you want to change the code in a runspace then you would clear it.

    Consider a runspace loaded with functions and using Invoke to execute a specific function.

    Your example of just adding the same code repeatedly is unrealistic.  Now that you see that you should be alright using runspaces.


    \_(ツ)_/

    Thursday, October 12, 2017 9:16 PM