Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save countzero/ca1fc4cbecc1676aa7e4d2433a8cdce1 to your computer and use it in GitHub Desktop.
Save countzero/ca1fc4cbecc1676aa7e4d2433a8cdce1 to your computer and use it in GitHub Desktop.
Execute PowerShell jobs in parallel with a max concurrency using the while loop polling strategy.
$maxConcurrentJobs = 5
$pollingFrequencyInMilliseconds = 50
for ($index=1; $index -le 23; $index++) {
while ($true) {
Get-Job -State Completed | Receive-Job | Remove-Job
$concurrencyLimitIsReached = $($(Get-Job -State Running).Count -ge $maxConcurrentJobs)
if ($concurrencyLimitIsReached) {
Write-Host "Reached concurrency limit of ${maxConcurrentJobs}, " -NoNewline
Write-Host "trying again in ${pollingFrequencyInMilliseconds} ms..."
Start-Sleep -Milliseconds $pollingFrequencyInMilliseconds
continue
}
$scriptBlock = {
Param (
[string] [Parameter(Mandatory=$true)] $id
)
$durationInMilliseconds = $(Get-Random -Minimum 500 -Maximum 1000)
Start-Sleep -Milliseconds $durationInMilliseconds
Write-Host "Job #${id} completed in ${durationInMilliseconds} ms."
}
Write-Host "Starting Job #${index}..."
Start-Job $scriptBlock -Name "Job #${index}" -ArgumentList $index | Out-Null
break
}
}
Get-Job | Receive-job -AutoRemoveJob -Wait
@countzero
Copy link
Author

This will output the job results directly after they are completed in the order in which they have finished:

Starting Job #1...
Starting Job #2...
Starting Job #3...
Starting Job #4...
Starting Job #5...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Job #1 completed in 634 ms.
Starting Job #6...
Reached concurrency limit of 5, trying again in 50 ms...
Job #2 completed in 669 ms.
Starting Job #7...
Job #3 completed in 505 ms.
Starting Job #8...
Reached concurrency limit of 5, trying again in 50 ms...
Job #4 completed in 695 ms.
Starting Job #9...
Job #5 completed in 642 ms.
Starting Job #10...
Job #6 completed in 535 ms.
Starting Job #11...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Job #7 completed in 673 ms.
Starting Job #12...
Job #8 completed in 631 ms.
Starting Job #13...
Job #9 completed in 589 ms.
Starting Job #14...
Job #10 completed in 516 ms.
Starting Job #15...
Job #11 completed in 613 ms.
Starting Job #16...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Job #12 completed in 561 ms.
Starting Job #17...
Job #13 completed in 505 ms.
Starting Job #18...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Job #14 completed in 662 ms.
Job #15 completed in 506 ms.
Starting Job #19...
Starting Job #20...
Job #16 completed in 671 ms.
Starting Job #21...
Reached concurrency limit of 5, trying again in 50 ms...
Reached concurrency limit of 5, trying again in 50 ms...
Job #17 completed in 661 ms.
Starting Job #22...
Job #18 completed in 614 ms.
Starting Job #23...
Job #19 completed in 531 ms.
Job #20 completed in 556 ms.
Job #21 completed in 606 ms.
Job #22 completed in 505 ms.
Job #23 completed in 505 ms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment