Skip to content

Instantly share code, notes, and snippets.

@pythonlessons

pythonlessons/transformers_training_9.py

Created September 4, 2023 15:04
Show Gist options
  • Save pythonlessons/f36560dc0bb4acf29f8e5bf8c5231dd5 to your computer and use it in GitHub Desktop.
Save pythonlessons/f36560dc0bb4acf29f8e5bf8c5231dd5 to your computer and use it in GitHub Desktop.
Download ZIP
transformers_training
Raw
transformers_training_9.py
# Create Training Data Provider
train_dataProvider = DataProvider(
train_dataset,
batch_size=configs.batch_size,
batch_postprocessors=[preprocess_inputs],
use_cache=True,
)
# Create Validation Data Provider
val_dataProvider = DataProvider(
val_dataset,
batch_size=configs.batch_size,
batch_postprocessors=[preprocess_inputs],
use_cache=True,
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment