Skip to content

Instantly share code, notes, and snippets.

@adgaudio
adgaudio / guided_filter_nd.py
Created March 17, 2021 10:18
Guided Filter supporting multi-channel guide image and 1 channel source image
"""
PyTorch Guided Filter for multi-channel (color) guide image and 1 channel
(grayscale) source image
"""
import torch as T
import torch.nn as nn
def box_filter_1d(tensor, dim, r):
cs = tensor.cumsum(dim).transpose(dim, 0)
@adgaudio
adgaudio / download_all_pretrained_pytorch_models.ipy
Created August 15, 2019 21:29
A quick hack script that uses ipython to download all pretrained pytorch models
# this is a quick ipython script to download all pre-trained pytorch models.
# run like this:
# ipython THIS_SCRIPT.ipy
import torchvision as tv, types, os.path
# get list of urls ... in a brittle way.
x = {k: v for dct in [getattr(y, 'model_urls') for y in (getattr(tv.models, x) for x in dir(tv.models)) if isinstance(y, types.ModuleType)] for k, v in dct.items()}
@adgaudio
adgaudio / tesseract_git_install.sh
Created February 21, 2018 21:31
Install tesseract and leptonica from source
#!/usr/bin/env bash
# This script installs leptonica and tesseract from source
# it does not install other pre-requisites to a custom location.
# side note: install prefix is defined once per library.
# side note: it clones git repositories in the current directory.
set -e
set -u
@adgaudio
adgaudio / DecayCounter.java
Last active December 19, 2017 20:16
A weighted counter that remembers the most frequent and recent pairs on a 2-color graph.
import java.util.HashMap;
/* A weighted counter that remembers most frequent and recent pairs on a 2-color graph, where:
* - any pair (a_i, b_i) contains elements a_i from set A and elements b_i are from set B. A and B are disjoint.
*
* This counter basically implements a recurrence relation to maintain scores for each pair:
* score = memory * prev_score + (1-memory) * (+/-)1
*
* "memory" is a value between 0 and 1 chooses how much history history to take into account.
*
// these double forward slashes are comments. you can write whatever you
// want in the text after the double slashes.
x = 10;
y = 20;
z = 5;
cube([x,y,z], center=true);
cube([x,y,z], center=false);
@adgaudio
adgaudio / distributed_percentile_algorithm.py
Last active August 29, 2015 14:05
Distributed Percentile and Distributed Median - a proof of concept and example
"""
This example demonstrates a distributed algorithm to identify the
percentile of a distributed data set.
Because this is a toy implementation, the data isn't actually
distributed across multiple machines.
"""
import numpy as np
@adgaudio
adgaudio / spark_serialization_demo.py
Last active October 24, 2016 23:25
This gist demonstrates that spark 0.9.1 (and I'm guessing also 1.0.0) don't serialize a logger instance properly when code runs on workers
"""
This gist demonstrates that spark 1.0.0 and 0.9.1
don't serialize a logger instance properly when code runs on workers.
run this code via:
spark-submit spark_serialization_demo.py
- or -
pyspark spark_serialization_demo.py
"""
import pyspark
@adgaudio
adgaudio / Demo - R Python Spark.ipynb
Last active August 29, 2015 14:01
Demo - R Python Spark.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@adgaudio
adgaudio / sparkR_install_notes.sh
Created April 26, 2014 05:00
SparkR install notes
# My SparkR install notes. SparkR gives R access to Apache Spark.
#
# For details about SparkR, see their site:
# http://amplab-extras.github.io/SparkR-pkg/
#
# Author: Alex Gaudio <adgaudio@gmail.com>
#
# Note the aweful hack where I symlink libjvm.so to /usr/lib. I did that to get rJava installed.
@adgaudio
adgaudio / starcluster_dns_update.py
Last active August 29, 2015 13:56
StarCluster Plugin: Configure Nodes with Dynamic DNS
"""A starcluster plugin that registers starcluster nodes in DNS
It assumes that nsupdate command can be run from the same machine where you run this plugin.
"""
from starcluster import clustersetup
import subprocess
from os.path import join
# You should configure these to your needs
DNS_ZONE = "example.com"