Skip to content

Instantly share code, notes, and snippets.

@iamaziz

iamaziz/LLM-from-scratch.ipynb

Created July 23, 2023 17:02
Show Gist options
  • Save iamaziz/171170dce60d9cd07fab221507fd1d52 to your computer and use it in GitHub Desktop.
Save iamaziz/171170dce60d9cd07fab221507fd1d52 to your computer and use it in GitHub Desktop.
Download ZIP
Building a large language model (LLM) from scratch (for learning and fun - inspired by Llama2).
Display the source blob
Display the rendered blob
Raw
LLM-from-scratch.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@AliHaider20
Copy link

You'll definitely get to know the structure of any model, such as Mistral, but obviously, you won't get the accuracy because SOTA models such as Mistral, Zephyr, Phi, Llama, etc are trained on a large amount of data which takes days of training on high end GPU's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment