Skip to content

Instantly share code, notes, and snippets.

@thomwolf

thomwolf/create_model_tokenizer.py

Last active May 3, 2019 08:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save thomwolf/379ae303ec982f2e8dc713180e1ffbc2 to your computer and use it in GitHub Desktop.
Save thomwolf/379ae303ec982f2e8dc713180e1ffbc2 to your computer and use it in GitHub Desktop.
Download ZIP
Instantiate OpenAI GPT model and tokenizer from pretrained checkpoint
Raw
create_model_tokenizer.py
from pytorch_pretrained_bert import OpenAIGPTDoubleHeadsModel, OpenAIGPTTokenizer
model = OpenAIGPTDoubleHeadsModel.from_pretrained('openai-gpt')
tokenizer = OpenAIGPTTokenizer.from_pretrained('openai-gpt')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment