Skip to content

Instantly share code, notes, and snippets.

@DocGarbanzo
Last active February 6, 2023 00:50
Show Gist options
  • Save DocGarbanzo/71ae8e894c305d03a7f9995ac6a430dd to your computer and use it in GitHub Desktop.
Save DocGarbanzo/71ae8e894c305d03a7f9995ac6a430dd to your computer and use it in GitHub Desktop.
A demo notebook for tensorflow datasets from generator

Creating training pipelines in tensorflow using generators

Here we show how using generators allows to use large data sets that don't fit into computer memory for training. We focus on models with multiple input and output arrays, where the generator has to provide the data in form of dictionaries to the tf.dataset. This data pipeline is used in the Donkey Car training. The notebook is simply extracting the pipeline mechanics to give a simplified view on the functionality.

Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment