Skip to content

Instantly share code, notes, and snippets.

@hammad93

hammad93/raspberry_pi_cm4_llm_metrics.csv

Last active May 4, 2024 21:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hammad93/eab28862024e480a2046d51bc39bd462 to your computer and use it in GitHub Desktop.
Save hammad93/eab28862024e480a2046d51bc39bd462 to your computer and use it in GitHub Desktop.
Download ZIP
Metrics collected from running ollama on a Raspberry Pi Computer Module 4
Raw
raspberry_pi_cm4_llm_metrics.csv
Model ID Total Duration Load Duration Prompt Eval Count Prompt Eval Duration Prompt Eval Rate Eval Count Eval Duration Eval Rate
phi3 1m 4.327686216s 6.086472ms 10 tokens 4.939021s 2.02 tokens/s 82 tokens 59.248218s 1.38 tokens/s
phi3 2m46.198225633s 5.833183ms 25 tokens 12.424108s 2.01 tokens/s 204 tokens 2m33.627457s 1.33 tokens/s
llama3 37.608249348s 6.886587ms 15 tokens 14.571366s 1.03 tokens/s 18 tokens 22.897199s 0.79 tokens/s
llama3 8m42.169986317s 5.151967ms 45 tokens 43.251874s 1.04 tokens/s 348 tokens 7m58.760736s 0.73 tokens/s
llama3:8b-instruct-q2_k 34.435642592s 6.87875ms 15 tokens 14.698864s 1.02 tokens/s 17 tokens 19.588981s 0.87 tokens/s
llama3:8b-instruct-q2_k 5m48.9021998s 6.390561ms 44 tokens 43.557413s 1.01 tokens/s 255 tokens 5m5.191865s 0.84 tokens/s
tinyllama 15.660182069s 4.394809ms 40 tokens 5.447574s 7.34 tokens/s 51 tokens 10.072085s 5.06 tokens/s
tinyllama 1m35.89412616s 7.999929ms 92 tokens 13.983224s 6.58 tokens/s 382 tokens 1m21.75818s 4.67 tokens/s
gemma:2b-instruct 20.357998127s 7.156725849ms 14 tokens 4.037678s 3.47 tokens/s 19 tokens 9.02777s 2.10 tokens/s
gemma:2b-instruct 47.30295752s 4.596759ms 25 tokens 6.892115s 3.63 tokens/s 82 tokens 40.266289s 2.04 tokens/s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment