Skip to content

Instantly share code, notes, and snippets.

@gngdb
gngdb / index_select_example.py
Last active November 20, 2018 17:21
index_select and then reshaping is faster than just indexing?
import torch
if __name__ == '__main__':
X = torch.randn(100)
out_shape = (100,100)
idxs = torch.randint(high=100, size=out_shape).long()
assert torch.abs(X[idxs] - X.index_select(0, idxs.view(-1)).view(*out_shape)).max() < 1e-3
from timeit import timeit
setup = 'import torch; X = torch.randn(100); out_shape=(100,100); idxs = torch.randint(high=100, size=out_shape).long()'
print("X[idxs]: ", timeit("_ = X[idxs]", setup=setup, number=100))
@gngdb
gngdb / hashing.py
Last active November 20, 2018 12:03
Inefficient HashedNet implementation: https://arxiv.org/abs/1504.04788
# implementation of https://arxiv.org/abs/1504.04788
import torch
import torch.nn as nn
import torch.nn.functional as F
import xxhash
class HashFunction(object):
"""Hash function as described in the paper, maps a key (i,j) to a natural number
in {1,...,K_L}"""
@gngdb
gngdb / example_usage.py
Last active May 18, 2022 05:32
Wrap PyTorch functions for scipy's optimize.minimize: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html (I also made a repo to do this https://github.com/gngdb/pytorch-minimize, although I had forgotten about this gist at the time)
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import numpy as np
from scipy import optimize
from obj import PyTorchObjective
@gngdb
gngdb / Efficient Top 1 Error Scatter Plot.ipynb
Last active February 5, 2023 17:19
Graphing performance for a report.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@gngdb
gngdb / reduce.py
Last active October 13, 2018 12:47
Trying to matmul reduce in PyTorch faster.
import torch
from functools import reduce
import math
def is_power(n):
# https://stackoverflow.com/a/29480710/6937913
n = n/2
if n == 2:
return True
elif n > 2:
@gngdb
gngdb / Interpolation with ReLUs in PyTorch.ipynb
Last active September 17, 2018 10:32
Interpolation with ReLUs in PyTorch
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@gngdb
gngdb / Least Squares in PyTorch.ipynb
Last active April 27, 2020 04:03
Least Squares in PyTorch
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@gngdb
gngdb / save_bits.py
Created August 27, 2018 10:48
Save a string of ones and zeros as those ones and zeros in binary using Python.
import random
from io import StringIO
def write_bitstream(fname, bits):
# bits are a string of ones and zeros, based on this
# stackoverflow answer: https://stackoverflow.com/a/16888829/6938913
# was broken due to utf-8 encoding using up to 4 bytes: https://stackoverflow.com/a/33349765/6937913
sio = StringIO(bits)
with open(fname, 'wb') as f:
while 1:
@gngdb
gngdb / logging.py
Created August 17, 2018 13:54
Log training using your progress bar object; after `pbar.update` call `pbar.log` with whatever you want to store as kwargs.
from tqdm import tqdm
class TrainingProgress(tqdm):
"""Make the progress bar store progress as it displays it when the log method is called."""
def log(self, *args, **kwargs):
if 'trace' not in self.__dict__.keys():
self.trace = {}
# store anything with an integer or float datatype in the trace dictionary
# with global index as the key
for k in kwargs:
@gngdb
gngdb / dct.py
Last active August 11, 2018 11:23
Inefficient DCTs in pytorch.
import math
import numpy as np
from scipy.fftpack import dct
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim