nvjpeg-test

README.md

test nvjpeg perfomance on server

you will need:

• nvidia driver
• CUDA
  • https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local
  • https://developer.nvidia.com/cuda-11.3.0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local
  • check version here: https://www.nvidia.com/Download/index.aspx?lang=en-us
• nvidia-docker

Installation

• clone this gist or download it with wget
• run this commands:

git clone https://gist.github.com/c4c002bd904af665c71bd02da31ac0eb.git
cd c4c002bd904af665c71bd02da31ac0eb/
chmod +x run.sh install.sh install_cuda_and_nvidia_driver.sh

• to install drivers/nvidia-docker/docker/docker-compose run: ./install.sh
• if all requirements are already installed run docker with ./run.sh

one-line for run

• git clone https://gist.github.com/c4c002bd904af665c71bd02da31ac0eb.git && cd c4c002bd904af665c71bd02da31ac0eb/&& chmod +x run.sh install.sh install_cuda_and_nvidia_driver.sh
• ./run.sh

Usage

You can edit test.sh file it will be executed inside a docker container

docker-compose.yaml

version: '3'

services:
  nvjpeg-test:
    container_name: "nvjpeg-test"
    image: registry.gitlab.com/neuro-core/admin/tools/nvjpeg-test:v0.0.2
    restart: always
    runtime: nvidia
    command: "test.sh 10"
    volumes:
      - ./test.sh:/workspace/test.sh
    environment:
      LOG_LEVEL: 'DEBUG'

install.sh

echo ""
echo "RUN THIS SCRIPT ONLY AFTER SUCCESSFULL INSTALLATION OF NVIDIA DRIVER AND CUDA"
echo ""
echo "=============================================================================="
echo ""
echo ""

echo "Installing system requirements"
sudo apt update && sudo apt upgrade -y
sudo apt install -y vim ffmpeg git net-tools curl wget make cmake gcc g++ linux-headers-$(uname -r) dkms build-essential linux-headers-generic

echo "Installing docker and docker-compose"
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
curl https://get.docker.com | sh && sudo systemctl --now enable docker

echo "Installing nvidia-docker"
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker
sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

echo "You need to auth on gitlab.com to download containers"
sudo docker login registry.gitlab.com

install_cuda_and_nvidia_driver.sh

echo "Installing system requirements"
sudo apt update && sudo apt upgrade -y
sudo apt install -y vim ffmpeg git net-tools curl wget make cmake gcc g++ linux-headers-$(uname -r) dkms build-essential linux-headers-generic

echo "Download Cuda and Nvidia Driver"
wget https://developer.download.nvidia.com/compute/cuda/11.5.1/local_installers/cuda_11.5.1_495.29.05_linux.run
sudo chmod +x cuda_11.5.1_495.29.05_linux.run

echo "Install Cuda"
sudo ./cuda_11.5.1_495.29.05_linux.run

echo "check logs if installation above failed"
echo "cat /var/log/cuda-installer.log"
echo "cat /var/log/nvidia-installer.log"

run.sh

### auth docker
# docker login registry.gitlab.com

# run container
docker-compose -f docker-compose.yaml up

test.sh

# you can paste here any commands you need to execute inside a docker container

#!/bin/bash
counter=$1
while [ $counter -gt 0 ]
do
   /usr/bin/python3 test_jpg.py
   counter=$(( $counter - 1 ))
done