Skip to content

Instantly share code, notes, and snippets.

@sgraaf
Last active January 20, 2020 16:58
Show Gist options
  • Save sgraaf/6e39f4b19e97bb308d28cef4805a3783 to your computer and use it in GitHub Desktop.
Save sgraaf/6e39f4b19e97bb308d28cef4805a3783 to your computer and use it in GitHub Desktop.
Tokenizers timing experiments: Multithreaded performance
implementation mean execution time
submit 1min 8s
map 1min 9s
encode_batch 10.6s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment