DocGarbanzo

Creating training pipelines in tensorflow using generators

Here we show how using generators allows to use large data sets that don't fit into computer memory for training. We focus on models with multiple input and output arrays, where the generator has to provide the data in form of dictionaries to the tf.dataset. This data pipeline is used in the Donkey Car training. The notebook is simply extracting the pipeline mechanics to give a simplified view on the functionality.

DocGarbanzo

Mock up of alternative donkeycar parts and car infrastructure.

Here we share a global data bus which allows to remove the explicit input and output arguments for current parts when added to the Vehicle. Each part runs in its own thread with its own frequency. Parts can write to and read from the data bus asynchronously and there is no global vehicle loop any longer. There is some resemblance to ROS whereby parts subscribe to topics. There is currently no control about type safety or allowed types in the data bus but that could be added easily.

DocGarbanzo

Python factory method

This is a simple approach for a factory that makes objects out of dictionaries. The only two ingredients are

• a metaclass Factory to manage the class registration and export a make function
• a base class Creatable that implements the creation funtion by calling the initializer

The user only has to derive classes from the base class Creatable and then has access to a dictionary-based creation of instances of such classes.

Please note

In the file where Factory.make() is called, it is necessary to import the Creatable concrete class otherwise the metaclass initialisation will not run.