install text generation web ui

install-text-generation-web-ui.md

1. Install CUDA

• https://developer.nvidia.com/cuda-downloads

2. set env var (.profile)

export PATH=/usr/local/cuda-12.2/compat:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/compat:$LD_LIBRARY_PATH

3. install miniconda

• https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html

4. install pytorch

• https://pytorch.org/get-started/locally/ or

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

• optional, checks that you have CUDA enabled

python -m torch.utils.collect_env 

5. clone Text Generation Web UI

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
python -m pip install -r requirements.txt

6. start web ui

python server.py