Running tasks parallel in Powershell

async_powershell.md

There are several ways to implement asynchronous processing in Powershell as below.

• background job
  • Background Jobs
  • multithreading - Can Powershell Run Comands in Parallel? - Stack Overflow
  • foreach - Running tasks parallel in powershell - Stack Overflow
  • リモート コンピューターのパラレル バッチ操作(Invoke-Command -AsJob)
• Powershell workflow
• using async, await
• RunspacePool
  • RunspacePoolを使って、PowerShellを非同期実行 - GMOインターネット株式会社
  • Using Background Runspaces Instead of PSJobs For Better Performance | Learn Powershell | Achieve More

Here is a sample code of Powershell by using PSJobs. However, it is very slow that it took about 30sec.

$start = Get-date

# Define what each job does
$block = {
  param($server)

  echo $server  # dummy command
}

# Loop throgh the server list
"server_aaa", "server_bbb", "server_ccc" | %{
  # Show the loop variable here is correct
  Write-Host "processing $_ ..." 

  # Execute the jobs in parallel
  Start-Job $block -ArgumentList $_ > $null
}

# Wait for all to complete
While (Get-Job -State "Running") {
  # Start-Sleep 1
}

# Display output from all jobs
Get-Job | Receive-Job

# Cleanup
Remove-Job *

$end = Get-date
'{0,-30} : {1,10:#,##0.00} ms' -f 'Time to run code', ($end - $start).TotalMilliseconds

Here is a sample code of Powershell by using background Runspaces jobs instead of PSJobs. It took about 0.2sec.

$start = Get-date

# Define what each job does
$block = {
  Param($server)

  echo $server  # dummy command
}

$array_ps  = New-Object System.Collections.ArrayList
$array_job = New-Object System.Collections.ArrayList

# Loop through the server list and save jobs to array
"server_aaa", "server_bbb", "server_ccc" | %{
  $ps = [PowerShell]::Create().AddScript($block).AddArgument($_)
  $job =$ps.BeginInvoke()

  [void] $array_ps.Add($ps)
  [void] $array_job.Add($job)
}

# execute jobs from array
$results = New-Object System.Collections.ArrayList
while ($array_ps.Count -gt 0) {
  for($i = 0; $i -lt $array_ps.Count; $i++) {
    $ps  = $array_ps[$i]
    $job = $array_job[$i]
    if($ps -ne $null) {
      if($job.IsCompleted) {
        [void] $results.Add($ps.EndInvoke($job))

        $ps.Dispose()
        $array_ps.RemoveAt($i)
        $array_job.RemoveAt($i)
        { break outer } > $null
      }
    }
  }
}

# Output the results of jobs
foreach($r in $results) {
  echo $r
}

$end = Get-date
'{0,-30} : {1,10:#,##0.00} ms' -f 'Time to run code', ($end - $start).TotalMilliseconds

References

• powershell - ArrayList Prints Numbers - Stack Overflow