PyTorch implementation of an autoencoder.

autoencoder-pytorch.ipynb

Thanks for sharing the notebook and your medium article!

Unfortunately it crashes three times when using CUDA, for beginners that could be difficult to resolve.
These issues can be easily fixed with the following corrections:

In code cell 8, change
test_examples = batch_features.view(-1, 784)
to
test_examples = batch_features.view(-1, 784).to(device)

In Code cell 9 (visualize results), change
plt.imshow(test_examples[index].numpy().reshape(28, 28))
to
plt.imshow(test_examples[index].cpu().numpy().reshape(28, 28))
and finally in code cell 9 as well, change
plt.imshow(reconstruction[index].numpy().reshape(28, 28))
to
plt.imshow(reconstruction[index].cpu().numpy().reshape(28, 28))

Thank you, @imrekovacs I shall edit this notebook accordingly.

Can you please confirm my understanding below :

by using "forward()" function, we are developing an autoencoder :
where encoder does have 2 layers both outputing 128 units and the reverse applicable to the decoder.

thanks very much for sharing

Thanks for sharing the notebook and your medium article!

Unfortunately it crashes three times when using CUDA, for beginners that could be difficult to resolve.
These issues can be easily fixed with the following corrections:

In code cell 8, change
test_examples = batch_features.view(-1, 784)
to
test_examples = batch_features.view(-1, 784).to(device)

In Code cell 9 (visualize results), change
plt.imshow(test_examples[index].numpy().reshape(28, 28))
to
plt.imshow(test_examples[index].cpu().numpy().reshape(28, 28))
and finally in code cell 9 as well, change
plt.imshow(reconstruction[index].numpy().reshape(28, 28))
to
plt.imshow(reconstruction[index].cpu().numpy().reshape(28, 28))

thanks for these amends - they work perfectly!

Can you please confirm my understanding below :

by using "forward()" function, we are developing an autoencoder : where encoder does have 2 layers both outputing 128 units and the reverse applicable to the decoder.

I didn't quite understand your question, but by forward method input traverse through the network with your mentioned architecture and give us an output which can be then used to calculate loss and optimize all weights through the optimization process (using calculated gradients from the last loss.backward method)

forward: tensor multiplication
model: uses forward to calculate output
loss: how far the generated output is from the original(target)
loss.backward: calculate gradients of every participated item in the existed neural network
optimizer.step: update every tensor (W, b) in the network

I tried running this and I got this error:

raceback (most recent call last):
File "/Users/brianweston/Documents/Stanford_CS/CS231N_CNN/Project/cs231n_final_project/scripts/training/train_FMNIST.py", line 109, in
for batch_features, _ in train_loader:
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in next
data = self._next_data()
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 570, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
return self.collate_fn(data)
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 172, in default_collate
return [default_collate(samples) for samples in transposed] # Backwards compatibility.
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 172, in
return [default_collate(samples) for samples in transposed] # Backwards compatibility.
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 180, in default_collate
raise TypeError(default_collate_err_msg_format.format(elem_type))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'PIL.Image.Image'>
(base) brianweston@MBP training % /Users/brianweston/miniforge3/bin/python /Users/brianweston/Documents/Stanford_CS/CS231N_CNN/Project/cs231n_final_project/scripts/training/train_FMNIST.py
Traceback (most recent call last):
File "/Users/brianweston/Documents/Stanford_CS/CS231N_CNN/Project/cs231n_final_project/scripts/training/train_FMNIST.py", line 109, in
for batch_features, _ in train_loader:
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in next
data = self._next_data()
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 570, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
return self.collate_fn(data)
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 172, in default_collate
return [default_collate(samples) for samples in transposed] # Backwards compatibility.
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 172, in
return [default_collate(samples) for samples in transposed] # Backwards compatibility.
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 180, in default_collate
raise TypeError(default_collate_err_msg_format.format(elem_type))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'PIL.Image.Image'>
(base) brianweston@MBP training % /Users/brianweston/miniforge3/bin/python /Users/brianweston/Documents/Stanford_CS/CS231N_CNN/Project/cs231n_final_project/scripts/training/train_FMNIST.py
Traceback (most recent call last):
File "/Users/brianweston/Documents/Stanford_CS/CS231N_CNN/Project/cs231n_final_project/scripts/training/train_FMNIST.py", line 60, in
ax.imshow(np.array(img), cmap='gist_gray')
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/matplotlib/_api/deprecation.py", line 456, in wrapper
return func(*args, **kwargs)
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/matplotlib/init.py", line 1412, in inner
return func(ax, *map(sanitize_sequence, args), **kwargs)
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/matplotlib/axes/_axes.py", line 5488, in imshow
im.set_data(X)
File "/Users/brianweston/miniforge3/lib/python3.8/site-packages/matplotlib/image.py", line 715, in set_data
raise TypeError("Invalid shape {} for image data"
TypeError: Invalid shape (1, 28, 28) for image data

Any idea what the fix is?

Kernel dies when running the Visualize Results, no error code, no changes. Everything prior to this successfully ran.