Skip to content

Instantly share code, notes, and snippets.

@sgraaf

sgraaf/transformers_vs_tokenizers.csv

Last active January 19, 2020 19:05
Show Gist options
  • Save sgraaf/c4a854424a265f43de726ce2b1ba1095 to your computer and use it in GitHub Desktop.
Save sgraaf/c4a854424a265f43de726ce2b1ba1095 to your computer and use it in GitHub Desktop.
Download ZIP
Tokenizers timing experiments: Transformers vs Tokenizers
Raw
transformers_vs_tokenizers.csv
implementation mean execution time
transformers 6min 42s
tokenizers 45.6s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment