Skip to content

Instantly share code, notes, and snippets.

@goldingn
Created October 31, 2017 22:19
Show Gist options
  • Save goldingn/409c2f97a86b335e91c204e9ad6b0721 to your computer and use it in GitHub Desktop.
Save goldingn/409c2f97a86b335e91c204e9ad6b0721 to your computer and use it in GitHub Desktop.
example of executing a zoon workflow in parallel (using experimental branch)
# install the experimental parallel branch
# remotes::install_github("zoonproject/zoon@parallel")
library (zoon)
# example workflow for 4 independent models that may take a while to run
run_wf <- function () {
workflow(occurrence = UKAnophelesPlumbeus,
covariate = UKBioclim,
process = Replicate(Background(n = 1000), 4),
model = GBM(max.trees = 10000),
output = SameTimePlaceMap)
}
# run it once to download the spatial data (needed for fair timings)
run_wf()
# sequential processing
system.time(run_wf())
# user system elapsed
# 82.241 0.950 129.140
# parallel processing (future should be installed with zoon)
library (future)
plan(multiprocess)
system.time(run_wf())
# user system elapsed
# 92.340 1.922 80.541
# turn off parallel
plan(sequential)
# you can use plan() to switch to differenct parallel backends e.g. to send jobs
# to run on remote compute clusters, using packages like future.batchtools.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment