Skip to content

Instantly share code, notes, and snippets.

View Multihuntr's full-sized avatar

Brandon Victor Multihuntr

  • La Trobe University
View GitHub Profile
@Multihuntr
Multihuntr / mat3.py
Last active January 7, 2022 01:31
Random Affine Crop in the style of Albumentations for a Rasterio Dataset with minimal dependencies
# Utility functions for managing 3x3 matrices for cv2.warpAffine in pure numpy
import numpy as np
def identity():
return np.eye(3, dtype=np.float64)
def affine(A=None, t=None):
@Multihuntr
Multihuntr / mean_avg_precision.py
Created July 28, 2021 11:36
Mean average precision for object detection
# Reimplementation of: https://github.com/aladdinpersson/Machine-Learning-Collection/blob/master/ML/Pytorch/object_detection/metrics/mean_avg_precision.py
# Now with more vectorisation!
def precision_recall_curve_th(is_tp, confs, n_true, eps=1e-8):
# Sort by confs
order = (-confs).argsort()
is_tp = is_tp[order]
confs = confs[order]
# Cumulative sum true positives and number of predictions
@Multihuntr
Multihuntr / partitioned.py
Created February 4, 2021 02:47
Sqlalchemy create partitioned tables in a for loop in Postgresql database
# BIG NOTE:
# I made this becuase I thought the method was interesting. It's not useful in it's current form.
# If you can, use an index instead. https://stackoverflow.com/a/27895337
from sqlalchemy import Column, Integer, Float, ForeignKey, event, DDL
from sqlalchemy.schema import CreateSchema
from sqlalchemy.dialects.postgresql import DOUBLE_PRECISION
from sqlalchemy.ext.declarative import declarative_base, declared_attr
Base = declarative_base()
@Multihuntr
Multihuntr / listlike.py
Created June 18, 2020 06:04
A list-like wrapper for dictionary-like objects
import collections
class ListLike(collections.abc.MutableSequence):
'''
A list-like interface wrapping dictionary-like objects.
Uses integer keys, like you would for a list, has range checking,
negative indexing and slicing working as you'd expect.
i.e.
with shelve.open('foo.dat') as sf:
@Multihuntr
Multihuntr / multi_level_index.py
Created May 7, 2020 03:01
For flat indexing into a nested structure without flattening that structure.
import bisect
import numpy as np
import collections.abc
class TwoLevelIndex(collections.abc.Sequence):
def __init__(self, coll):
counts = [len(thing) for thing in coll]
self.cum_counts = np.cumsum(counts)
def __getitem__(self, idx):
seg_idx = bisect.bisect(self.cum_counts, idx)
@Multihuntr
Multihuntr / wrapper.py
Created April 24, 2020 10:38
Generic Pytorch Module Wrapper - When nn.Sequential just isn't enough
# I keep properties on my main nn.Modules. e.g. a list of the training statistics the model is tracking.
# I wanted to perform a set of extra actions across multiple different modules without having to
# - write those steps into each of the 5+ different model definitions, or
# - explicitly expose those values on the wrapper module.
# It's fairly trivial, but if you don't use the try: super(), it doesn't keep the `wrapped` property.
import torch
import torch.nn as nn
class Wrapper(nn.Module):
@Multihuntr
Multihuntr / derive.py
Last active December 17, 2019 06:10
Decorator for lazy-loaded derived values
def derive(_from, _coll='_derived', name=None):
'''
Creates a decorator that caches derived values.
Utilises a property on the object to keep a collection of derived properties,
but requires the calling obj to manage that collection (clearing, instantiation, etc).
Args:
_from (str): Property this property is derived from
_coll (str, optional): Collection name of derived property names
name (str, optional): Overwrite the property used to cache
@Multihuntr
Multihuntr / pytorch_tensor_to_image.py
Created September 2, 2019 03:35
One liner to save a GPU pytorch tensor to disk as an image using PIL
from PIL import Image
# Assumes tensor is shaped [c, h, w]
Image.fromarray(tensor.detach().cpu().numpy().transpose([1, 2, 0])).save('test.png')
# Assumes tensor is shaped [n, c, h, w]
Image.fromarray(tensor[0].detach().cpu().numpy().transpose([1, 2, 0])).save('test.png')
# Yeah, pretty simple, but annoying to remember...
@Multihuntr
Multihuntr / gauss_smooth.py
Created August 2, 2019 07:15
Numpy gaussian smoothing
# Super simple 1D smoothing with just numpy. Just smooths a few sharp edges.
import math
import numpy as np
def gaussian_kernel(n=5):
x = np.arange(n)/n
g = math.e ** -(x**2/((n - 2)**2)) # drop the constant factor at the start
return g/g.sum() # normalise so it sums to 1
@Multihuntr
Multihuntr / minimal_nvvl_tests.py
Created June 11, 2018 03:45
Minimal examples of using NVIDIA/nvvl
# Simplest case (note: VideoDataset.__get__ puts frames on CPU)
import nvvl
d = nvvl.VideoDataset(['prepared.mp4'], 3)
fr = d[0]
print(type(fr))
print(fr.shape)
# Custom processing (note: VideoDataset.__get__ puts frames on CPU)
import nvvl
d = nvvl.VideoDataset(['prepared.mp4'], 3, processing={'a': nvvl.ProcessDesc()})