Skip to content

Instantly share code, notes, and snippets.

@novasush

novasush/parallelize_extraction.py

Created March 7, 2020 18:06
Show Gist options
  • Save novasush/cbf26c3905d782bbf51bf8e74800e31f to your computer and use it in GitHub Desktop.
Save novasush/cbf26c3905d782bbf51bf8e74800e31f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
parallelize_extraction.py
# Parallelize the extraction of the stored TFRecords of
# the cats_vs_dogs dataset by using the interleave operation with
# cycle_length = 4 and the number of parallel calls set to tf.data.experimental.AUTOTUNE.
train_dataset = files.interleave(tf.data.TFRecordDataset,
cycle_length=4,
num_parallel_calls=tf.data.experimental.AUTOTUNE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment