AmgadHasan

import os
import time
import argparse

start = time.time()

os.system("nvidia-smi")

# import libraries

AmgadHasan

How to install nvidia drivers + nvidia cuda on new vms

https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=runfile_local

1. Download runfile:

from the command line, run:

wget https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda_11.7.1_515.65.01_linux.run

AmgadHasan

# Create a virtual environment named myenv
python3 -m venv myvenv
# Activate this venv
source myvenv/bin/activate
# Now the venv is activated, install the packages
pip install transformers ctranslate2
ct2-transformers-converter \
    --model whisper-large-v2
    --output_dir whisper-large-v2-ct2 \
    --copy_files tokenizer_config.json \

AmgadHasan

Overview

Sometimes we get data that resembles an opened file and we can't handle it easily. An example of this is if you download a wav audio file directly into memory, so that the binary contents of the file are already stored in an object. One way to handle it is to write the binary contents to an actual file on disk. However, this is inefficient.

Another way is to create a file-like object that can be dealt with as normal files.

In this example, we have the binary contents of a wav audio file and we would like to decode it to extract the audio array and sampling rate. We want to do this all in-memory without writing to disk.

1. Dependencies

We will be doing this using python.

AmgadHasan

Motivation

Many people want to get starting with using chatbots locally. However, they often are intimidated by the complexity and the steep learning curve needed to even run a basic chatbot.

In this gist, I share a few simple methods to run a chatbot locally without the need to do 69 installation steps. All you need a machine with enough memory to run the model. A computer with 8GB of cpu ram is the minimum requirement.

These are currently the recommneded methods.

1. Docker Containers