Installation of vLLM v0.8.2 using Pytorch v2.6.0 (CPU!) and CPython 3.12 using a Nix developer shell:
- Activate developer shell:
nix develop(seeflake.nixbelow) - Build vLLM
- Switch to tmp dir:
cd /tmp/ - Create venv:
python -m venv vllm- ensure it uses the
pythonbinary from the developer shell
- ensure it uses the
- Switch to tmp dir:
- Activate venv:
source /tmp/vllm/bin/activate