Installation of vLLM v0.8.2 using Pytorch v2.6.0 (CPU!) and CPython 3.12 using a Nix developer shell:
- Activate developer shell:
nix develop
(seeflake.nix
below) - Build vLLM
- Switch to tmp dir:
cd /tmp/
- Create venv:
python -m venv vllm
- ensure it uses the
python
binary from the developer shell
- ensure it uses the
- Switch to tmp dir:
- Activate venv:
source /tmp/vllm/bin/activate