Skip to content

Instantly share code, notes, and snippets.

@jamesmaino

jamesmaino/r-plumber-on-aws.md

Last active November 16, 2022 01:31
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jamesmaino/8a19db39547e0f88bc855df45fa638e7 to your computer and use it in GitHub Desktop.
Save jamesmaino/8a19db39547e0f88bc855df45fa638e7 to your computer and use it in GitHub Desktop.
Download ZIP
Create a Plumber API for R code hosted on AWS EC2
Raw
r-plumber-on-aws.md

Deploying an API for an R code base with {plumber} and AWS

This post is largely built around a tutorial by Martin Lukac but this tutorial did not set up a persistent API service so some final steps are adapted from the plumber documentation.

Setup the AWS instance

You start by logging into your AWS account. Once in, you select an RStudio Amazon Machine Image (AMI) from this link. You are free to choose any, but it's more efficient to choose based on your geography. The Rstudio service is actually not really necesary, as all the steps can be done directly from the terminal, but some users might feel more comfortable with it.

Once in AWS, you have to select the machine you want to run it on. AWS EC2 has a free tier option called t2.micro, and that's the one I will be using here. Click Review and Launch.

In the next step, we need to modify how we will be able to access the instance. Find 'Security Groups' and click Edit security groups. We will add a couple of gates that we will use to connect to the instance:

  • Add "HTTP"
  • Add "Custom TCP Rule", change Port Range to "8000" — this will be a way to access your API
  • Add "Custom ICMP Rule IPv4", change Protocol "Echo Request", and add Source "0.0.0.0/0" - this will be to check whether everything works well and to troubleshoot. This can be later removed when not necessary.

You can confirm your selections and move further. You will be asked about an SSH key pair_. This is a tricky one: This window creates a file that will be used to access the EC2 server from your Terminal. We will need to do this to set up some things later. You can use an existing key pair if you have one (but since you are following this tutorial, it is more likely that you don't). Select Create a new key pair and name it (e.g. "my-plumber-key"). This will initiate a download of a new file "my-plumber-key.pem". For now, you can leave it in "Downloads" (if you are on Mac), but later it would be better to store it somewhere safe. Hit Launch!

If you are still confused or something does not work well, try following this guide: https://jagg19.github.io/2019/08/aws-r/. Sometimes it takes a bit of trial and error to get things right.

Now that your instance is successfully running, we need to log in via Terminal and install some missing pieces that will allow us to run an API on R and expose it to the world. Go to your AWS console and find your new EC2 (running) instance. Click on the instance number (a long number looking like this: i-098a9123e1c53...a12ds. Find a Connect button and click it. Open the SSH client tab and follow these instructions:

  • Open Terminal (or any other SSH client)
  • Navigate to the location of the key "my-plumber-key.pem" that you downloaded above.
  • Run this command: chmod 400 my-plumber-key.pem
  • Connect to your instance with the following command looking like this one: ssh -i "rstudio-plumber.pem" ubuntu@ec*-*-*-**-**.eu-west-*.compute.amazonaws.com. You can directly copy this command from the AWS website in the Connect --> SSH section.

Now that you are connected to your AWS EC2 instance, we need to install some missing components. The {plumber} package relies on a library called Libsodium for encryption, decryption, signatures, and password hashing. Ubuntu does not have this installed by default, so we need to download and install it. Follow these steps:

  • Get a link to the most current stable version (at the time of the writing, it is libsodium-1.0.17-stable.tar.gz) on this page. The link I will use is this https://download.libsodium.org/libsodium/releases/libsodium-1.0.17-stable.tar.gz. It might change, so make sure you adjust it.
  • Go back to the terminal and run wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.17-stable.tar.gz (substituting your link). This will download the file to the hard drive of your EC2 instance.
  • You can check that if it happened correctly by running the ls command, which will print the contents of the current directory. It should contain a libsodium file --- e.g. libsodium-1.0.17-stable.tar.gz, in my case.
  • The file we downloaded is compressed, so we need to extract it by running tar -xf libsodium-1.0.17-stable.tar.gz (again, substitute your file name in case you are using a different version).
  • Again, you can check if there is a new folder by running ls. If you see it there, you can enter it by running cd libsodium-stable.
  • When you are in, execute the following commands to build the code from source:

./configure
make && make check
sudo make install
sudo ldconfig

  • It will take a while to install, but when everything finishes, run a final command sudo apt install libsodium-dev. If this fails, see the following page for troubleshooting. Sometimes Ubuntu decides to start updating some system files in the background and will prevent you from finishing the installation.

I ended up disabling the auto update service and rebooting with the following commands.

sudo systemctl disable --now apt-daily.timer

...reboot ...try again

sudo apt install libsodium-dev
sudo apt update
sudo apt full-upgrade
sudo systemctl enable apt-daily.timer

Install plumber.

sudo R 
install.packages("plumber")
library(plumber)

Make things happen in R

If everything worked, you can log into your R session by going to the Public IPv4 DNS link in your instance summary (You can get this by going back to the AWS console, finding a list of your EC2 instances, and clicking on your running instance). It looks similar to this: http://ec2-*-*-***-*.eu-west-2.compute.amazonaws.com (again with numbers instead of *).

Your login is "rstudio" and the password is your Instance ID, looking like this i-23g0d124q24698123. Change your password right away --- there is guidance in the document that you see in front of you. It's worth it for security reasons.

Create a new folder called plumber-apis and in it two new scripts my-plumber.R and run-my-plumber.R.

In my-plumber.R, add the following. It does not matter at this stage what the functions are. You can add all your functions later. Let's make it work first:

# my-plumber.R

#* Echo back the input
#* @param msg The message to echo
#* @get /echo
function(msg=""){
  list(msg = paste0("The message is: '", msg, "'"))
}

#* Plot a histogram
#* @png
#* @get /plot
function(){
  rand <- rnorm(100)
  hist(rand)
}

#* Return the sum of two numbers
#* @param a The first number to add
#* @param b The second number to add
#* @get /sum
function(a, b){
  as.numeric(a) + as.numeric(b)
}

Orginally, this final function made a post request but since 1.1.0 plumber has implemented a method warning which appears when making a http request through the browser with innappropirate headers. Tools like postman can be used for testing.

# run-my-plumber.R
library(plumber)

root <- pr(paste0(getwd(), "/my-plumber.R"))
pr_run(root, host = "0.0.0.0", port = 8000)

In the Terminal in RStudio, at the bottom, a tab right next to the Console type cd plumber-apis to get into the folder where you have your scripts and run R CMD BATCH run-my-plumber.R.

Open the plumber-apis folder in the Files viewer on the right. You should see a new file called run-my-plumber.Rout. When you open it, you will see:

> pr_run(root, host = "0.0.0.0", port = 8000)
Running plumber API at http://0.0.0.0:8000
Running swagger Docs at http://127.0.0.1:8000/__docs__/

Test the link in the browser with: http://ec2-*-*-***-*.eu-west-2.compute.amazonaws.com:8000/sum?b=1&a=3.

The API works but to ensure it is a persistent service that automatically restarts on any server resets some further steps are necessary.

systemd

systemd is the service manager used by certain Linux distributions including RedHat/CentOS 7, SUSE 12, and Ubuntu versions 16.04 and later. See other options here. Docker and Docker compile are likely to be the most reproducible and flexible approaches, but the following works for smaller projects.

If you use a Linux server you can use systemd to run Plumber as a service that can be accessed from your local network or even outside your network depending on your firewall rules. This option is similar to using the Docker method. One of the main advantages of using systemd over using Docker is that systemd won’t bypass firewall rules (Docker does!) and avoids the overhead of running a container.

Compared to plumber::do_provision() this option won’t create a new droplet if you use DigitalOcean; it will run on your existing droplet instead.

To implement this option you’ll complete the following three steps from the terminal:

  • Verify that you have the plumber package available globally on the server:

R -e 'install.packages("plumber", repos = "https://cran.rstudio.com/")'

  • Run sudo nano /etc/systemd/system/plumber-api.service, then paste and adapt this content:
[Unit]
Description=Plumber API
# After=postgresql
# (or mariadb, mysql, etc if you use a DB with Plumber, otherwise leave this commented)

[Service]
ExecStart=/usr/bin/Rscript run-my-plumber.R
Restart=on-abnormal
WorkingDirectory=/home/rstudio/plumber-apis

[Install]
WantedBy=multi-user.target
  • Activate the service (for auto-start on power/reboot) and start it:
sudo systemctl enable plumber-api  # automatically start the service when the server boots
sudo systemctl start plumber-api   # start the service right now

To check if your API is running, type systemctl | grep running in the terminal and should display plumber-api.service \ loaded active running Plumber API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment