Setup CUDA on AWS EC2 Instance This tutorial is based on this AWS tutorial. In this tutorial, we will install Nvidia driver on AWS EC2 instance and compile and run llama.cpp on it. Create an AWS EC2 instance Here we use g5.4xlarge instance with Ubuntu 22.04 AMI, which use Nvidia A10G GPU. Install Nvidia driver