Skip to content

Instantly share code, notes, and snippets.

@LysandreJik LysandreJik/loading_data.py Secret
Created Oct 8, 2019

Embed
What would you like to do?
Loading data using tensorflow datasets
import tensorflow_datasets
from transformers import glue_convert_examples_to_features
data = tensorflow_datasets.load("glue/mrpc")
train_dataset = data["train"]
validation_dataset = data["validation"]
train_dataset = glue_convert_examples_to_features(train_dataset, bert_tokenizer, 128, 'mrpc')
validation_dataset = glue_convert_examples_to_features(validation_dataset, bert_tokenizer, 128, 'mrpc')
train_dataset = train_dataset.shuffle(100).batch(32).repeat(2)
validation_dataset = validation_dataset.batch(64)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.