Skip to content

Instantly share code, notes, and snippets.

@LysandreJik

LysandreJik/BERT_keras.py

Last active October 24, 2019 18:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save LysandreJik/9df5034d53944f9a7d88c6eefa8b4d00 to your computer and use it in GitHub Desktop.
Save LysandreJik/9df5034d53944f9a7d88c6eefa8b4d00 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
BERT_keras.py
model = TFBertForSequenceClassification.from_pretrained("bert-base-cased")
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
data = tensorflow_datasets.load("glue/mrpc")
train_dataset = data["train"]
train_dataset = glue_convert_examples_to_features(train_dataset, tokenizer, 128, 'mrpc')
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])
model.fit(train_dataset, epochs=3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment