This is an example for how to run a highly parallel job on a slurm cluster. Mostly this is a copy of the original documentation (https://cran.r-project.org/web/packages/rslurm/vignettes/rslurm.html) but with some extra options added and a lot of words removed. To use this script. Begin with module load R/4.2.3 and then open R with the R command.…

rslurm_example.Rmd

---
title: "rslurm example workflow"
author: "Mark Ziemann"
date: "`r Sys.Date()`"
output:
  html_document:
    toc: true
    toc_float: true
    fig_width: 7
    fig_height: 7
theme: cosmo
---

## Introduction

Here we are trying to leverage HPC to speed up simulatons in parallel.

```{r,libs}

library("rslurm")

```

## Make a function to be run in parallel on the HPC

```{r,func1}

test_func <- function(par_mu, par_sd) {
    samp <- rnorm(10^6, par_mu, par_sd)
    c(s_mu = mean(samp), s_sd = sd(samp))
}

```

## Parameters for running

Create a dataframe with parameters to run.

```{r,params}

params <- expand.grid("par_mu"=1:10,"par_sd"=seq(0.1, 1, length.out = 10))

head(params, 3)

```

## Run on slurm HPC

Now run some jobs.

Setting aside 5 GB memory per node.

```{r,run1}

sopt1 <- list(time = '1:00:00', mem="5G")

sjob <- slurm_apply(test_func, params, jobname = 'test_apply',
  nodes = 10, cpus_per_node = 2, submit = TRUE,slurm_options = sopt1)

sjob


```

The job will be submitted and the results can be found in the folder called `_rslurm_test_apply`.

## Retrieve the results

```{r,collect1}

res <- get_slurm_out(sjob, outtype = 'table', wait = FALSE)

str(res)

res

```

## Clean up the intermediate files


```{r,clean1}

cleanup_files(sjob)

```

## Session information

```{r,session}

sessionInfo()

```