Notes from Moving R programs to Rivanna workshop, 10/17/19

Moving_R_programs_to_Rivanna.md

Moving R programs to Rivanna notes

Workshop date: 10/17/2019

Acessing Rivanna

Home directory: 50 Mb storage
Scratch: 10 TB (90 day limit)
Can purchase storage; requires a PTAO

To get on Rivanna, see:
https://www.rc.virginia.edu/userinfo/rivanna/allocations/

Once on Rivanna, best to submit code to compute nodes. They run whether you are logged in or not.

Two ways to connect to Rivanna:

1. Open OnDemand (web interface)
2. ssh client

To use Open OnDemand:

• go to https://www.rc.virginia.edu/userinfo/rivanna/login/
• Click Launch Open OnDemand
• Log in with NetBadge Credentials
• Go to Interactive Apps and select FastX Web
• Log in again with NetBadge Credentials
• Select MATE and click Launch Session; select MATE again and select Launch (be sure your browser allows pop-ups for this site)
• this launches a desktop environment in your browser; black box icon opens terminal

If you want to use an ssh client, RC recommends MobaXterm for Windows (free home version)

Setting up R environment

Probably want to use the same version of R that you have on your latop. To see what version you have on your laptop, enter R.version.string in the R console.

Use modules to choose R version.
Open terminal in FastX desktop.
Use module spider R to see what versions are available.
If you want to use R/3.6.1, enter module spider R/3.6.1 to see other modules you need to load.
Enter those modules before R. Order matters. For example:

module load gcc/7.1.0 openmpi/3.1.4 R/3.6.1 rstudio/1.0341

If it works, there will be no echo or confirmation. Also, sometimes a module load can be slow (up to 5 minute wait)

When loading RStudio, enter & (space &) at the end of the module load command to allow access to the terminal while RStudio runs.

In RStudio, you can load the packages you need. If it asks to create a local library, answer yes.

To see what packages you have installed on your laptop: rownames(installed.packages())

We can run stuff in RStudio, but we really should submit to compute node.

Copying Files to Rivanna

Using Open OnDemand:

• Go to Files, Home Directory
• Click the upload button in the upper right corner

Using FastX Desktop:

• Open FireFox
• Go to UVa Box
• Download files

Can use git clone if you want to clone a repo to your Home Directory

Can also use Globus. See https://www.rc.virginia.edu/userinfo/globus/

Running R programs on Rivanna

Use Rscript to submit R script from command line:
Example: Rscript hello.R

Use a SLURM script to request resources to run code. The SLURM script will also contain the request to run R code.

SLURM documentation: https://www.rc.virginia.edu/userinfo/rivanna/slurm/

Example SLURM script:

#!/bin/bash
#SBATCH nodes=1               #total number of nodes for the job
#SBATCH ntasks =1             #how many copies of code to run
#SBATCH time=1 12:00:00       #amount of time for the whole job
#SBATCH partition=standard    #the queue/partition to run on
#SBATCH account= myGroupName  #the account/allocation to use

module purge
module load gcc R  #load modules that my job needs
Rscript myProg.R   #command line execution of my job

Can write a SLURM script in RStudio; save as a txt with .slurm extension

Run a slurm script in the terminal. Example:
-bash 4.1$ sbatch job_script.slurm

To display the status of only your active jobs, type:
squeue -u <your_user_id>

To delete a job from the queue, use the scancel command with the job ID number at the command line prompt:
-bash 4.1$ scancel 18316

SLURM script needs to be in same directory as R scripts.

Can also use RStudio Server:

• go to Open OnDemand
• Interactive App...RStudio Server
• populate fields (need to populate "allocation" field)

To shut down click Delete

Help: hpc-support@virginia.edu