mikebohdan/README.md

## README.md

      
    Raw
  

              README.md
            
          
Setup Tabby on WSL

What is tabby? Tabby is a free, locally running alternative for GitHub Copilot. If you have an Nvidia GPU,
you can run it with significant acceleration. Currently, it supports only a few programming languages, but if
you use one of these, you can use all the benefits of AI-assisted code completion for free. You can choose
different models with different sizes to better fit your hardware.
Preparations

First of all, make sure that you are running Windows 10 or newer, have the latest drivers for your GPU,
and have installed WSL 2. This tutorial uses Ubuntu 22.04.2 LTS as a guest OS.
On guest OS, you need to install the following things:

NVidia Toolkit
git
rust

NVidia toolkit

Make sure you pick the right version of the toolkit. Then follow the instructions.

Building a local executable file

First, ensure all listed below are installed correctly and your system is ready.

Rust is installed, and $HOME/.cargo/bin in your path
Nvidia toolkit is installed, and you have a cuda compiler /usr/local/cuda/bin/nvcc

Getting Tabby's code

Follow the instructions in the Contributing section.
Building executable

Tabby is using the Cuda compiler to build a part requiring GPU acceleration so make sure you have the CUDACXX environment variable
set before you start to compile.
export CUDACXX=/usr/local/cuda/bin/nvcc
Now, we can build an executable like this:
cargo build --features cuda --release --package tabby
After the build, you can copy your executable to your preferred location. For example:
cp target/release/tabby ~/.local/bin/
Make sure $HOME/.local/bin in your PATH.
Running

Now you can run the Tabby server.
tabby serve --model TabbyML/StarCoder-3B --device cuda
Other models

If this model is too big or your hardware can handle a larger one, go to the
models registry tutorial to pick the right one.
Known issues


/usr/lib/wsl/lib/libcuda.so.1 is not a symbolic link - solution