Skip to content

Instantly share code, notes, and snippets.

@theycallmeloki
theycallmeloki / Raspberry Pi 4 x 6 cluster 65B Llama runner gist
Last active May 28, 2024 11:43
an ansbile script that quickly bootstraps a Pi 4 cluster with tools required to run a 65B llama model
You will need the following two files, be sure to change the IP address in both sections, the `hostfile_contents`
as well as the ansible inventory section to reflect your configuration
Rest of the steps should run parallely across the different machines, for any updates to this script,
check this thread: https://github.com/ggerganov/llama.cpp/issues/2164#issuecomment-1639640846
If you want an automated way to do this in the future and would like an opionated setup procedure,
check this repo: https://github.com/theycallmeloki/edith-cli