AWS EMR bootstrap to install R packages from CRAN
This bootstrap is useful if you want to deploy
SparkR applications that run arbitrary code on the EMR cluster's workers. The
R code will need to have its dependencies already installed on each of the workers, and will fail otherwise.
This is the case if you use functions such as
How to use the bootstrap
- You will first have to download the gist to a file and then upload it to S3 in a bucket of your choice.
- Using the AWS EMR Console create a cluster and choose advanced options.
- In Step 3 you can configure your bootstraps. Choose to Configure and add a Custom action
- For the Name you can fill something like Install CRAN dependencies
- For the Script location you will need to point to where you have uploaded the gist (Eg.
- As Optional arguments you can add the following:
--packages- Where you list all of the CRAN packages that you depend on, separated by semicolon. Eg:
Other interesting material
Take a look at my other