Instantly share code, notes, and snippets.

# Jan Hünermann janhuenermann

Last active May 22, 2021
Two-dimensional positional encoding in PyTorch (inspired by https://arxiv.org/abs/1706.03762)
View positional_encoding_2d.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch from typing import Tuple, Optional @torch.jit.script def positional_encoding_2d(shape: Tuple[int, int, int], temperature: float = 1e4, scale: float = 2*math.pi, dtype: Optional[torch.dtype] = None, device: Optional[torch.device] = None): """Returns the two-dimensional positional encoding as shape [d_model, h, w]""" d_model, h, w = shape[-3:] i = torch.arange(d_model // 4, dtype=dtype, device=device)
Last active May 21, 2021
Compute the area of a polygon
View shoelace.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 def shoelace(poly): """Returns the area of the polygon using the shoelace formula""" s1 = np.sum(poly[:,0] * np.roll(poly[:,1], -1)) s2 = np.sum(poly[:,1] * np.roll(poly[:,0], -1)) return np.absolute(s1 - s2) / 2.
Last active Nov 4, 2021
Find the intersection of two polygons in NumPy using the Sutherland–Hodgman algorithm
View polygon-intersection.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 # Line intersection utility # Works with broadcasting def intersect_lines(line1, line2): """Computes the intersection of line1 and line2, both described by tuple of origin and direction vector""" p, r = line1 q, s = line2 rs = np.cross(r, s)[..., None] d = (q - p) / (rs + 1e-24) t = np.cross(d, s)[..., None]
Last active May 21, 2021
Find the smallest rotated rectangle that covers a given polygon
View min-rotated-rect.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import numpy as np # Gift wrapping algorithm, from https://en.wikipedia.org/wiki/Gift_wrapping_algorithm def find_convex_hull(poly): """ Returns the convex hull of the given polygon Can be replaced using OpenCV: ``` hull = cv2.convexHull(poly, clockwise=True, returnPoints=False)
Last active Jun 28, 2021
Tiny Attention (https://arxiv.org/abs/2105.08050)
View tiny-attention.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 from torch import nn, chunk, einsum from torch.nn.functional import softmax from math import sqrt class TinyAttention(nn.Module): def __init__(self, d_attn: int, d_ffn: int): super().__init__() self.proj_qkv = nn.Linear(d_ffn, 3 * d_attn) self.proj_ffn = nn.Linear(d_attn, d_ffn)
Last active May 21, 2021
Efficient implementation of Generalized Intersection over Union (GIoU) in PyTorch, as in https://giou.stanford.edu/
View giou.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch @torch.jit.script def generalized_box_iou(boxes1, boxes2, strict: bool = False): """ Generalized IoU from https://giou.stanford.edu/ The boxes should be in [x0, y0, x1, y1] format Returns a [*, N, M] pairwise matrix, where N = boxes1.size(-2) and M = boxes2.size(-2) """
Last active Aug 21, 2021
Sinkhorn Optimal Transport Algorithm in PyTorch
View sinkhorn.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch @torch.jit.script def log_optimal_transport(Z, iters: int): m, n = Z.shape log_mu = -torch.tensor(m).to(Z).log().expand(Z.shape[:-2] + [m]) log_nu = -torch.tensor(n).to(Z).log().expand(Z.shape[:-2] + [n]) u, v = torch.zeros_like(log_mu), torch.zeros_like(log_nu) for _ in range(iters): v = log_nu - torch.logsumexp(Z + u.unsqueeze(-1), dim=-2)
Created May 9, 2018
TensorFlow using MNIST. Using standard TF APIs, in 30 lines of code.
View mnist2.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data session = tf.Session() mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) input = tf.placeholder(tf.float32, shape=(None, 784)) x = tf.layers.dense(input, 128, activation=tf.nn.relu) # layer 1 x = tf.layers.dense(x, 128, activation=tf.nn.relu) # layer 2 x = tf.layers.dense(x, 10, activation=None) # layer 3
Last active May 9, 2018
TensorFlow using MNIST. Example of how to implement SGD and fully-connected layers.
View mnist.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) def fullyConnected(x, name, input_size, output_size=64): with tf.variable_scope(name): kernel = tf.get_variable(name="kernel", initializer=tf.random_normal(shape=(input_size, output_size), stddev=0.1))
Last active Apr 15, 2017
A little program that calculates poker probabilities.