Skip to content

Instantly share code, notes, and snippets.

@NMZivkovic

NMZivkovic/load_process.py

Last active February 2, 2020 16:36
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save NMZivkovic/78bc99d0d7c375a0ce2aa2d88f173fbf to your computer and use it in GitHub Desktop.
Save NMZivkovic/78bc99d0d7c375a0ce2aa2d88f173fbf to your computer and use it in GitHub Desktop.
Download ZIP
Raw
load_process.py
def load_process(self, shuffle_size = 1000):
self.loaded_dataset = self.dataset.map(self._load_labeled_data, num_parallel_calls=tf.data.experimental.AUTOTUNE)
self.loaded_dataset = self.loaded_dataset.cache()
# Shuffle data and create batches
self.loaded_dataset = self.loaded_dataset.shuffle(buffer_size=shuffle_size)
self.loaded_dataset = self.loaded_dataset.repeat()
self.loaded_dataset = self.loaded_dataset.batch(self.batch_size)
# Make dataset fetch batches in the background during the training of the model.
self.loaded_dataset = self.loaded_dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment