Skip to content

Instantly share code, notes, and snippets.

Created Oct 23, 2019
What would you like to do?
class Layer(Function):
Abstract model of a neural network layer. In addition to Function, a Layer
also has weights and gradients with respect to the weights.
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.weight = {}
self.weight_update = {}
def _init_weights(self, *args, **kwargs):
Initializes the weights.
def _update_weights(self, lr):
Updates the weights using the corresponding _global_ gradients computed during
lr: float. Learning rate.
for weight_key, weight in self.weight.items():
self.weight[weight_key] = self.weight[weight_key] - lr * self.weight_update[weight_key]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment