Skip to content

Instantly share code, notes, and snippets.

@tysam-code
tysam-code / discrete-action-backprop-bypass-hlb-cifar10-demo.py
Last active March 14, 2024 09:24
This is a network sketch. Network sketches aren't complete, released versions, but rather a (hopefully convincing) proof-of-concept of a particular idea. The intent of this network sketch is to show that traditional backprop is not required in order to learn and coordinate a network's dataflow structure over a simulated discrete, non-backproppab…
# Sketch-specific note: a roughly ~25 run battery for this code estimated a roughly ~93.11% accuracy in the same number of steps as the baseline network, ~1.7x runtime overhead (much of which goes to the torch.randn allocations and extra layer calculations).
# Note: The one change we need to make if we're in Colab is to uncomment this below block.
# If we are in an ipython session or a notebook, clear the state to avoid bugs
#"""
try:
_ = get_ipython().__class__.__name__
## we set -f below to avoid prompting the user before clearing the notebook state
%reset -f
except NameError: