Skip to content

Instantly share code, notes, and snippets.

@pcuenca

pcuenca/gemma-tokenizer-test.ipynb

Created May 8, 2024 15:10
Show Gist options
  • Save pcuenca/917353acb55ba3d3347b21e82e91926e to your computer and use it in GitHub Desktop.
Save pcuenca/917353acb55ba3d3347b21e82e91926e to your computer and use it in GitHub Desktop.
Download ZIP
Gemma Tokenizer Tests
Display the source blob
Display the rendered blob
Raw
gemma-tokenizer-test.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment