Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
start = time.time()
# tokenize wikitext
encoded_output = tokenizer.encode_batch(text)
end = time.time()
print(end - start)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.