Open Termux and perform first time setup:
termux-change-repo
This opens a menu to make selections:
- Select
Mirror groupRotate
. - Choose the mirror group for your country.
- Use the arrows to move up and down.
- Use the spacebar to make selections.
- Use the Enter key to confirm selection.
Upgrade packages:
pkg upgrade -y
If there are any prompts to make a selection, press Enter for the default choice.
pkg install -y clang wget cmake git
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release
llama.cpp requires you to download a compatible model file:
- The model must be in GGUF format.
- The size of the model must be
3B
parameters or thereabouts. It needs to fit in the RAM requirements of the phone.
wget https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q6_K.gguf
./build/bin/llama-server -m Llama-3.2-3B-Instruct-Q6_K.gguf
The server will start listening at http://127.0.0.1:8080
. Minimize
the Termux app and it will continue running in the background.
-
Go to Settings
- Ollama Settings
- API URL
- Enter the URL:
http://localhost:8080/
(trailing/
is important) - Leave other settings as default.
- Enter a System Prompt if you wish.
- Ollama Settings
-
Start a New Chat.
-
Ask a question.