Skip to content

Instantly share code, notes, and snippets.

@novasush

novasush/parallelize_loading.py

Created March 7, 2020 18:15
Show Gist options
  • Save novasush/6c5beef3c496a71a17ba56b79248442f to your computer and use it in GitHub Desktop.
Save novasush/6c5beef3c496a71a17ba56b79248442f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
parallelize_loading.py
# Shuffle and batch the train_dataset. Use a buffer size of 1024
# for shuffling and a batch size 32 for batching.
train_dataset = train_dataset.shuffle(1024).batch(32)
# Parallelize the loading by prefetching the train_dataset.
# Set the prefetching buffer size to tf.data.experimental.AUTOTUNE.
train_dataset = train_dataset.prefetch(tf.data.experimental.AUTOTUNE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment