Asked by:
RunSpace Double Ups

Question
-
Hey I'm just getting into really using runspaces to gather information from the network to speed along the process but I'm noticing something intriguing with doubling up of information so I have a few questions about the script I'm using, in hopes you lovely people can help clarify things I'm clearly not understanding.
Apologies the script has gotten a little dirty as I'm trying to break it apart and test sections.
Question 1: the "$RunspacePool = [runspacefactory]::CreateRunspacePool(1,$breakdown)" would work out to say (1,4) variable my understanding is that should mean maximum threads of 4? however I notice the chunk is actually 5 threads, if I up the count its always 1 more and I'm not sure why.
Question 2: I noticed the last thread double ups on me if I were to run the below script against 20 PCs and expect results like:
Machine Username Disabled
Computer1 Administrator Active
Computer1 Guest Disabled
Computer2 Administrator Active
Computer2 Guest Disabled
Computer3 Administrator Active
Computer3 Guest Disabled
Computer4 Administrator Active
Computer4 Guest Disabled
Computer5 Administrator Active
Computer5 Guest Disabledbut instead I'll get the results of:
Machine Username Disabled
Computer1 Administrator Active
Computer1 Guest Disabled
Computer2 Administrator Active
Computer2 Guest Disabled
Computer3 Administrator Active
Computer3 Guest Disabled
Computer4 Administrator Active
Computer4 Guest DisabledComputer2 Administrator Active
Computer2 Guest Disabled
Computer5 Administrator Active
Computer5 Guest Disablednoting that the last thread runs again on a previously used thread and has the previously collated data still within in it to return. also thing to note when the script runs the next chunk despite the:
$jobs[$jobstart].PowerShell.Dispose()
$jobs[$jobstart].powershell = $null
$jobs[$jobstart].handle = $null
and
$jobs.clear()
$jobs = $null
lines let alone the $job variable being re declared I notice each thread is still retaining the previously returned information from prior jobs. how can I clear this so the threads do not retain or return the previous information again without having to "unique" the results table after the fact.
Example Script:
$Computers = Get-ADComputer -Filter * -Properties lastLogonTimestamp, operatingsystem, servicePrincipalName | ?{$_.enabled -eq $true} | ?{($_.servicePrincipalName -join "-") -notlike "*clustervirtual*"}
$breakdown = [int]$env:NUMBER_OF_PROCESSORS
$runner = $Computers.Count
$num1 = 0
$num2 = $breakdown
$check = 0
$Results =$null
#region Runspace Pool
[runspacefactory]::CreateRunspacePool()
$SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$RunspacePool = [runspacefactory]::CreateRunspacePool(1,$breakdown)
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
$RunspacePool.Open()
#endregion
do{
write-host "running block $num1..$num2 out of $runner" -ForegroundColor green
$jobs = New-Object System.Collections.ArrayList
$count = 1
ForEach($Computer in $Computers[$num1..$num2]){
$Parameters = @{
Computer = $computer
}
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$PowerShell.AddScript({
Param ($computer)
run custom script
return $Report
})
[void]$PowerShell.AddParameters($Parameters)
$Handle = $PowerShell.BeginInvoke()
$temp = '' | Select PowerShell,Handle
$temp.PowerShell = $PowerShell
$temp.handle = $Handle
[void]$jobs.Add($Temp)
}
$waittimer = 0
do{
if($jobs.handle.IsCompleted -like $false){
Write-Host "." -NoNewline
Start-Sleep -Seconds 1
$waittimer++}ELSE{$waittimer = 100}
}until($waittimer -eq 100)
$jobs = $jobs| ?{$_.handle.IsCompleted -like $true}
$jobcount = $jobs.Count
$jobstart = 0
do{
if($jobstart -lt $jobcount-1){
$Results += $jobs[$jobstart].powershell.EndInvoke($jobs[$jobstart].handle)
$jobs[$jobstart].PowerShell.Dispose()
$jobs[$jobstart].powershell = $null
$jobs[$jobstart].handle = $null
}ELSE{
$Results += $jobs[$jobstart].powershell.EndInvoke($jobs[$jobstart].handle) | ?{$_.machinename -like (($jobs[$jobstart].powershell.EndInvoke($jobs[$jobstart].handle) | select Machinename -Unique)[1]).machinename}
$jobs[$jobstart].PowerShell.Dispose()
$jobs[$jobstart].powershell = $null
$jobs[$jobstart].handle = $null
}
$jobstart++
}until($jobstart -eq $jobcount)
$jobs.clear()
$jobs = $null
$num1 = $num1+$breakdown+1
$num2 = $num2+$breakdown+1
if($num2 -ge $runner){$num2 = $runner
$check++}
}until($check -gt 1)
$RunspacePool.Close()
New-TimeSpan -Start $start -End (get-date)Friday, July 19, 2019 4:28 AM
All replies
-
A runspacepool can have a maximum number of runspaces and a minimum number of runspaces. The maximum is never exceeded. The minimum is used to prevent stalling when releasing large numbers of tasks. This ensures that there will always be a runspace available for a new task. It also allows tasks to be queued. The minimum should not exceed the number of cores available or overhead will be increased. RUnspaces are most useful on multi-core/processor systems.
I recommend that you use either jobs or a workflow. It is easier to manage and does not require systems programming experience.
Workflows are better managed for most PS tasks and are more efficient than jobs.
Note that a runspace is not a thread. There is a thread factory and thread creation API for threads. Runspaces are completely different. They are objects that are part of the PowerShell "Automation" namespace.
\_(ツ)_/
Friday, July 19, 2019 5:06 AM -
I get a lot of that and I've answered my question on question 1 with my coding its more on the seond part I'm stumped and I'd really like to get my head around it. Lets say I move the "$jobs = New-Object System.Collections.ArrayList" line to the top of the script so it is created only once the out put for $jobs.PowerShell | ft Streams, InstanceID reads as
Streams InstanceId
------- ----------
System.Management.Automation.PSDataStreams 7908e81f-5bf0-42ff-8df2-bf2954d49609
System.Management.Automation.PSDataStreams 98099023-5f63-4c97-b2d2-f96e37b1a431
System.Management.Automation.PSDataStreams f4c60c09-a678-4290-8aa4-8215cc84fd95
System.Management.Automation.PSDataStreams 5abcd4f4-9437-4d25-9051-74e0b17cadd7
System.Management.Automation.PSDataStreams f7c5e008-9f77-4ba3-bb62-0589b3ef2022
System.Management.Automation.PSDataStreams 2dac7fed-8c1d-4c79-a512-2390d2bda51d
System.Management.Automation.PSDataStreams 94e65d7f-4ee4-489d-9d8e-60e4e9846986
System.Management.Automation.PSDataStreams 113bbf7e-9f99-482f-a720-b8e799187587
System.Management.Automation.PSDataStreams f922dbc3-11ea-4ac1-a336-c70eafbaa37b
System.Management.Automation.PSDataStreams 1dd985cb-4ee3-4c56-a925-2897f6b15b3a
System.Management.Automation.PSDataStreams 225cffe4-87c2-47c9-a55b-8c41a9fdd0d4
System.Management.Automation.PSDataStreams 22a4e00c-b58d-4145-a586-7e63227e269a
System.Management.Automation.PSDataStreams 2f1ef13f-6a5d-45ac-8cf8-efcba987c702
System.Management.Automation.PSDataStreams f4abe787-6b84-4af1-9a14-99614af45163
System.Management.Automation.PSDataStreams a67cd3b0-9301-48ea-83de-987bf4382d9c
System.Management.Automation.PSDataStreams 49c9341f-b05d-4e94-9d3a-cb540ff158d7
System.Management.Automation.PSDataStreams fb13b982-fcdd-49dc-bb8f-167784a83ff4So each object within jobs as created a unique powershell instance then when you cycle through each job the first 4 (had a maximum of 4 against the createrunspacepool) look good they each have 1 computer against them and are returning a table of data of things my custom script ran to gather. but then we hit the 5th object in the jobs list and I can see that the computer from the fourth job shows in the fifth jobs table as well as the new expected computer i check the 6th job and same deal except it has the 3rd jobs PC plus the new expected one. and if I go to the 9th it as the 3rd, 4th and its new expected computer in the jobs outputs.
As I type this I realised I can shortcut the output issue by moving the $return value to being completely out of the do loop at the end (do loop was there while I was trying to break it all down and understand the code) and enter this code at the very end of the script.
$Returns = $jobs[(($jobs.Count)-$breakdown-1)..$jobs.Count] | ForEach {
$_.powershell.EndInvoke($_.handle)
$_.PowerShell.Dispose()
}Which is all fine and gets me the results I wish for but what is the point in all the other jobs and is there a more efficient way so when I run this against 10,000 devices I'm not creating so many instances.
Once Cleaned up the new script working looks like this for anyone else in a similar issue:
$start = Get-Date
$Computers = Get-ADComputer -Filter * -Properties lastLogonTimestamp, operatingsystem, servicePrincipalName | ?{$_.enabled -eq $true} | ?{($_.servicePrincipalName -join "-") -notlike "*clustervirtual*"}
$breakdown = [int]$env:NUMBER_OF_PROCESSORS
$Results =$null
#region Runspace Pool
[runspacefactory]::CreateRunspacePool()
$SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$RunspacePool = [runspacefactory]::CreateRunspacePool(1,$breakdown)
$RunspacePool.Open()
#endregion
$jobs = New-Object System.Collections.ArrayList
ForEach($Computer in $Computers){
$Parameters = @{
Computer = $computer
}
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$PowerShell.AddScript({
Param ($computer)
Custom Script
return $Report
})
[void]$PowerShell.AddParameters($Parameters)
$Handle = $PowerShell.BeginInvoke()
$temp = '' | Select PowerShell,Handle
$temp.PowerShell = $PowerShell
$temp.handle = $Handle
[void]$jobs.Add($Temp)
}
$waittimer = 0
do{
if($jobs.handle.IsCompleted -like $false){
$jobs.handle | ft
Write-Host "." -NoNewline
Start-Sleep -Seconds 1
$waittimer++}ELSE{$waittimer = 100}
}until($waittimer -eq 100)
$Returns = $jobs[(($jobs.Count)-$breakdown-1)..$jobs.Count] | ForEach {
$_.powershell.EndInvoke($_.handle)
}
$jobs| %{$_.PowerShell.Dispose()}
$jobs.clear()
$RunspacePool.Close()
New-TimeSpan -Start $start -End (get-date)Friday, July 19, 2019 7:16 AM -
Perhaps if you would post your code correctly we might be able to understand your issue.
How to post code in Technet Forums
How to ask questions in a technical forum
\_(ツ)_/
Friday, July 19, 2019 8:16 AM