Setup EC2 for RL OpenAI Gym

install.md

These notes mostly follow a tutorial on installing NVIDIA and CUDA to avoid collisions with X servers.

https://medium.com/@DJVJallday/a-how-to-on-deep-reinforcement-learning-setup-aws-with-keras-tensorflow-openai-gym-and-jupyter-88bc0cc67e02

Backup Links:

• https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html
• forgets to discuss the --overide --no-opengl-libs when installing cuda: https://gist.github.com/wangruohui/df039f0dc434d6486f5d4d098aa52d07

Open questions:

• Do I need to know xinit or startx?

Steps

1. Launch an EC2 Instance

   I started with the image Deep Learning Base AMI (Ubuntu), but since we're reinstalling CUDA anyway (with special options), you could probably start with any recent Ubuntu AMI.

2. Install NVIDIA drivers and CUDA from scratch, with --no-opengl-files options.

   Follow steps at https://gist.github.com/colllin/d5c016fb0c7d482aca9d4b5204e01abb.

   Deep Learning Base AMI comes with multiple CUDA versions installed. If we didn't need to mock a display, we could simply update the cuda symlink to point to the one we need.

   $ ls -g /usr/local/cuda
   $ sudo rm /usr/local/cuda
   $ sudo ln -s /usr/local/cuda-9.2 /usr/local/cuda
   

3. (Optional) Install python 3.6

   $ sudo add-apt-repository ppa:jonathonf/python-3.6
   $ sudo apt-get update
   $ sudo apt-get install python3.6
   

   Note: To run OpenAI baselines, you will need:

   $ sudo apt-get install python3.6-dev
   

4. Install steamcmd

   $ sudo dpkg --add-architecture i386
   $ sudo apt-get update
   $ sudo apt-get install steamcmd
   

5. (Optional) Install pipenv

   $ pip3 install pipenv
   

6. Setup repo, install steam ROMs...

   Follow: https://contest.openai.com/details

7. (Optional) Install pyvirtualdisplay to manage xvfb sessions for you!

   $ pipenv install pyvirtualdisplay
   

   Then, at the top of your python script:

   from pyvirtualdisplay import Display
   display = Display(visible=0, size=(1400, 900))
   display.start()