calling-dt-parallel.R

fn <- function() {
  FUN <- function(x) {
    data.table::getDTthreads()
  }
  parallel::mclapply(1:2, FUN, mc.cores = 2)
}

# Call `getDTthreads()` from the workers
# Should return 1 because we forked
fn()

# Now start it up in a separate process,
# then do the mclapply call which creates
# two workers to run getDTthreads() in.
# The two workers should be forks of the process, so
# why is this 6?
callr::r(fn)

Result when run in rstudio, locally, one line at a time. Not using reprex here because it runs everything in a separate R session

> fn <- function() {
+   FUN <- function(x) {
+     data.table::getDTthreads()
+   }
+   parallel::mclapply(1:2, FUN, mc.cores = 2)
+ }
> 
> # Call `getDTthreads()` from the workers
> # Should return 1 because we forked
> fn()
[[1]]
[1] 1

[[2]]
[1] 1

> 
> # Now start it up in a separate process,
> # then do the mclapply call which creates
> # two workers to run getDTthreads() in.
> # The two workers should be forks of the process, so
> # why is this 6?
> callr::r(fn)
[[1]]
[1] 6

[[2]]
[1] 6

See ?getDTthreads for documentation about data.table switching to use 1 thread in forked workers