Skip to content

Instantly share code, notes, and snippets.

View russelldc's full-sized avatar

Daniel Russell russelldc

View GitHub Profile
@russelldc
russelldc / opt.py
Created September 13, 2021 04:01
A few random optimizers I've come across
import math
from typing import TYPE_CHECKING, Any, Callable, Optional
import torch
import torch.nn.functional as F
import torch.nn as nn
from functools import reduce
import os
import torch.distributed as dist
from torch.optim.optimizer import Optimizer, required
import collections

Keybase proof

I hereby claim:

  • I am russelldc on github.
  • I am danielrussell (https://keybase.io/danielrussell) on keybase.
  • I have a public key ASDn0LMUeydPvAwGXR4F2j0TRsSmBt4cN895tPqc5F3Yjgo

To claim this, I am signing this object: