Skip to content

Instantly share code, notes, and snippets.

@tvatter
Last active October 18, 2018 20:16
Show Gist options
  • Save tvatter/2fcf3a9a99c256f9b9360f596b300715 to your computer and use it in GitHub Desktop.
Save tvatter/2fcf3a9a99c256f9b9360f596b300715 to your computer and use it in GitHub Desktop.
library(parallel)
## small/large obviously depend on the OS and memory available
n_small <- 1e2
n_large <- 1e4
## example from ?svd
hilbert <- function(n) { i <- 1:n; 1 / outer(i - 1, i, "+") }
my_svd <- function(..., n){
X <- hilbert(n)[, 1:6]
s <- svd(X)
return(TRUE)
}
mclapply(1:2, my_svd, n = n_small, mc.cores = 1) # works fine
mclapply(1:2, my_svd, n = n_small, mc.cores = 2) # works fine
mclapply(1:2, my_svd, n = n_large, mc.cores = 1) # works fine
mclapply(1:2, my_svd, n = n_large, mc.cores = 2) # returns a list of NULL
@tvatter
Copy link
Author

tvatter commented Aug 11, 2018

The multisession version definitely works, but I wondered about forked workers on linux/mac.

@HenrikBengtsson
Copy link

The multisession version definitely works, but I wondered about forked workers on linux/mac.

multicore (!= multisession) workers (e.g. plan("multicore")) use forked processes utilizing the same framework as parallel::mclapply() does internally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment