Step by step in starting a deep learning project (pytorch)

guide.md

Supervised learning

• Create the Dataset object and test if it can load the data correctly
• Create the transformation (in DataLoader) for the Dataset
• Create a model and test if the DataLoader can feed the data to the model
• Create the training procedure

Tips

Datasets

• Check the input and output distribution. Ideally they are as close as possible to normal distribution. If not, then apply transformation to make it close to normal distribution.
• If you can load the dataset in the memory, please do so. It could speed up the training quite significantly.

Models

• I usually check if untrained model gives reasonably random output (e.g. equal chance of being positive or negative).
• Find the model that could overfit the training data, then apply regularization.