Skip to content

Instantly share code, notes, and snippets.

@Birch-san

Birch-san/falcon-40b-spqr.md

Created June 9, 2023 22:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Birch-san/2371d1ec8d419063fe823f7dd55afafc to your computer and use it in GitHub Desktop.
Save Birch-san/2371d1ec8d419063fe823f7dd55afafc to your computer and use it in GitHub Desktop.
Download ZIP
Run Falcon-40B with 3.35-bit quantization via SpQR
Raw
falcon-40b-spqr.md

Instructions are a work-in-progress (I haven't managed it yet, just writing what I do as I go along).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment