Skip to content

Instantly share code, notes, and snippets.

@rain-1
Last active May 2, 2024 13:39
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rain-1/3f780d55d37c12f22a9d2dbfcc39ee52 to your computer and use it in GitHub Desktop.
Save rain-1/3f780d55d37c12f22a9d2dbfcc39ee52 to your computer and use it in GitHub Desktop.
On Llamafile

On Llamafile not making sense

The LLamafile project doesn't make sense.

The claim is that it is "bringing LLMs to the people", but you could already run an LLM - which is a large binary file containing lots of floating point numbers - by using llama.cpp.

Llamafile joins a compiled binary program to run LLMs with a weights binary into a single file. This isn't a useful goal. you could simply distribute a zip containing an .exe and a weights file together. Or better still: Decouple the program that runs these chatbots from the chatbot weights.

Imagine if PNG files were also an executable that could pop open a window that displays a PNG on your computer. There is a reason we don't do this: It's not good engineering.

There is a claim that llamafile is simpler or more convenient, because it's one thing. But it's actually not hard to download two things: an .exe for windows or mac or whatever and an LLM binary.

And honestly having your executable contain so much random looking binary data does not seem like a good choice in terms of security either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment