Skip to content

Instantly share code, notes, and snippets.

@NMZivkovic
Last active February 2, 2020 16:36
Show Gist options
  • Save NMZivkovic/78bc99d0d7c375a0ce2aa2d88f173fbf to your computer and use it in GitHub Desktop.
Save NMZivkovic/78bc99d0d7c375a0ce2aa2d88f173fbf to your computer and use it in GitHub Desktop.
def load_process(self, shuffle_size = 1000):
self.loaded_dataset = self.dataset.map(self._load_labeled_data, num_parallel_calls=tf.data.experimental.AUTOTUNE)
self.loaded_dataset = self.loaded_dataset.cache()
# Shuffle data and create batches
self.loaded_dataset = self.loaded_dataset.shuffle(buffer_size=shuffle_size)
self.loaded_dataset = self.loaded_dataset.repeat()
self.loaded_dataset = self.loaded_dataset.batch(self.batch_size)
# Make dataset fetch batches in the background during the training of the model.
self.loaded_dataset = self.loaded_dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment