Skip to content

Instantly share code, notes, and snippets.

@kvasilopoulos
Last active April 10, 2024 08:27
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kvasilopoulos/d49499ea854541924a8a4cc43a77fed0 to your computer and use it in GitHub Desktop.
Save kvasilopoulos/d49499ea854541924a8a4cc43a77fed0 to your computer and use it in GitHub Desktop.
How to include progressbar with doParallel (previously done only with the doSNOW-package) and foreach loop. Note, that you can easily wrap this into a function.
library(doParallel)
# Choose number of iterations
n <- 100
# Progress combine function
f <- function(iterator){
pb <- txtProgressBar(min = 1, max = iterator - 1, style = 3)
count <- 0
function(...) {
count <<- count + length(list(...)) - 1
setTxtProgressBar(pb, count)
flush.console()
cbind(...) # this can feed into .combine option of foreach
}
}
# Start a cluster
cl <- makeCluster(detectCores(), type='SOCK')
registerDoParallel(cl)
# Run the loop in parallel
result <- foreach(i = icount(n), .combine = f(n)) %dopar% {
rnorm(10)
}
## Or without the iterators dependency
# result <- foreach(i = 1:n, .combine = f(n)) %dopar% {
# rnorm(10)
#}
#Stop the cluster
stopCluster(cl)
@Spiritspeak
Copy link

For me this did not work - the progress bar remained invisible until the very end, when it suddenly reached 100%. See the slow code below:

result <- foreach(i = icount(n), .combine = f(n)) %dopar% {
  mean(rnorm(10000000))
}

Or:

result <- foreach(i = icount(n), .combine = f(n)) %dopar% {
  Sys.sleep(1)
  rnorm(10)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment