Skip to content

Instantly share code, notes, and snippets.

View Cyanogenoid's full-sized avatar

Yan Zhang Cyanogenoid

View GitHub Profile
@rtqichen
rtqichen / pytorch_weight_norm.py
Last active May 11, 2023 06:58
Pytorch weight normalization - works for all nn.Module (probably)
## Weight norm is now added to pytorch as a pre-hook, so use that instead :)
import torch
import torch.nn as nn
from torch.nn import Parameter
from functools import wraps
class WeightNorm(nn.Module):
append_g = '_g'
append_v = '_v'
@hrldcpr
hrldcpr / tree.md
Last active May 1, 2024 00:11
one-line tree in python

One-line Tree in Python

Using Python's built-in defaultdict we can easily define a tree data structure:

def tree(): return defaultdict(tree)

That's it!