Run llama2.c with WasmEdge

README.md

Run llama2.c with WasmEdge

Prepare wasi-sdk

export WASI_VERSION=20
export WASI_VERSION_FULL=${WASI_VERSION}.0
wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_VERSION}/wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
tar xvf wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
export WASI_SDK_PATH=`pwd`/wasi-sdk-${WASI_VERSION_FULL}
CC="${WASI_SDK_PATH}/bin/clang --sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"

Compile llama2.c to wasm

git clone https://github.com/karpathy/llama2.c.git
cd llama2.c
$CC run.c -D_WASI_EMULATED_PROCESS_CLOCKS -D_WASI_EMULATED_MMAN -lwasi-emulated-process-clocks -lwasi-emulated-mman -o run.wasm

Compile and run with WasmEdge

$ wasmedge compile run.wasm run-aot.wasm
[2023-07-24 16:39:52.851] [info] compile start
[2023-07-24 16:39:52.858] [info] verify start
[2023-07-24 16:39:52.862] [info] optimize start
[2023-07-24 16:39:53.251] [info] codegen start
[2023-07-24 16:39:53.608] [info] output start
[2023-07-24 16:39:53.611] [info] compile done
[2023-07-24 16:39:53.611] [info] output start

$ wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
$ wasmedge --dir .:. run-aot.wasm stories15M.bin
<s>
One day, a little fish named Fin was swimming in the sea. He saw a big, pretty reef. The reef was full of other fish, and they were all different. Fin wanted to find something else to play with.
Fin asked a big fish, "Do you know where I can find something fun?" The big fish, who was an ignorant boy, said, "Yes, I know where to find a fun place to swim under this reef. Follow me!"
Fin and the big fish swam together, and they had lots of fun. They swam and played all day. The other fish were happy they decided to follow the big fish to find something new.
<s>
Once upon a time, there was a little boy named Tim. He was very adventurous. Tim loved to play outside and get dirty. One day, his mom told him it was time to wash his clothes.
Tim went to his room and saw his mom's big smile. "What do you want to do, Tim?" she asked. Tim thought for a moment and said, "I want to play outside, but it's very dirty!" His mom smiled and said, "You can play in the mud after you was
achieved tok/s: 31.003040