Run Llama.cpp on Android in Termux

android-termux-llama.cpp.md

Run llama.cpp chat bot LLM on Pixel phone (Termux)

Requirements

• Android device with 4+ GB RAM
• Install Fdroid app
• Install Termux app
• Install GPTMobile app

Setup Termux

Open Termux and perform first time setup:

termux-change-repo

This opens a menu to make selections:

• Select Mirror groupRotate.
• Choose the mirror group for your country.
  • Use the arrows to move up and down.
  • Use the spacebar to make selections.
  • Use the Enter key to confirm selection.

Upgrade packages:

pkg upgrade -y

If there are any prompts to make a selection, press Enter for the default choice.

Install build dependencies

pkg install -y clang wget cmake git

Build llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release

Download the model

llama.cpp requires you to download a compatible model file:

• The model must be in GGUF format.
• The size of the model must be 3B parameters or thereabouts. It needs to fit in the RAM requirements of the phone.

I found this one that works.

wget https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q6_K.gguf

Run the server

./build/bin/llama-server -m Llama-3.2-3B-Instruct-Q6_K.gguf

The server will start listening at http://127.0.0.1:8080. Minimize the Termux app and it will continue running in the background.

Open the GPTMobile app

• Go to Settings

  • Ollama Settings
    • API URL
    • Enter the URL: http://localhost:8080/ (trailing / is important)
    • Leave other settings as default.
    • Enter a System Prompt if you wish.

• Start a New Chat.

• Ask a question.