Skip to content

Instantly share code, notes, and snippets.


James Melville jlmelville

View GitHub Profile

June 28 2023 O frabjous day! After much fruitless occasional faffing around with cmake settings over the course of several months, new CUDA update for WSL Ubuntu means that I am now able to build the nearest neighbors library Faiss with GPU support for my 1080 laptop graphics card. Here is the setup that worked for me.

Why would you want to build your own Faiss? Well, if you are using pip to install faiss-gpu, you can't get anything more recent than version 1.7.2. Also, with recent dependency management changes with newer Pythons (particularly Python 3.11), some packages are going to be playing catch-up with getting up to parity, so it may be useful to build from source. I don't remember how bad it was trying to install faiss-gpu on Python 3.11 with pip, but it must have been fairly un-fun, because I gave up and scuttled back to the relatively welcoming environs of Python 3.10 pretty quickly after briefly dipping my

jlmelville /
Last active September 5, 2022 17:28
Fixing the legacy gpg keyring warning when installing CUDA on Ubuntu for WSL2
jlmelville /
Last active December 24, 2021 20:19
20 newsgroups python/R
jlmelville / transitive_dependencies.R
Created November 30, 2021 16:39
transitive dependencies in R
View transitive_dependencies.R
# modified slightly from
distrib.pkgs <- library(lib.loc=R.home("library"))$results[,1]
dependencies <-
function(pkg, dependencies = c("Depends", "Imports", "LinkingTo"),
pl = installed.packages())
if (!(pkg %in% rownames(pl))) stop("unknown pkg ", pkg)
fields <- pl[pkg, dependencies]
jlmelville / revdepcheck.R
Last active November 29, 2021 05:04
Testing reverse dependencies for R packages
View revdepcheck.R
# may need to install bioconductor
if (!require("BiocManager", quietly = TRUE)) {
BiocManager::install(version = "3.14")
jlmelville /
Last active November 11, 2021 05:48
reading CSV into Pandas but also pickling it at the same time
import os
import pickle
from pathlib import Path
def read_data(dataset, suffix=None, dir=None, repickle=False):
"""Set `repickle=True` to rewrite pickle file if CSV has changed"""
home = str(Path.home())
if dir is None:
dataset_path = os.path.join(home, "dev", "datasets")
jlmelville / zca_whiten.R
Created May 6, 2020 02:30
ZCA whitening
View zca_whiten.R
whitening_matrix <- function(X, eps = 1e-8, zca = FALSE) {
X <- scale(X, center = TRUE, scale = FALSE)
s <- svd(X, nu = 0)
Linvsqr <- sqrt(nrow(X) - 1) / (s$d + eps)
W <- Linvsqr * t(s$v)
if (zca) {
W <- s$v %*% W
jlmelville / smallvis_bench.R
Created May 3, 2020 23:43
useful functions for running smallvis on multiple datasets
View smallvis_bench.R
benchmark <- function(datasets = smallvis_datasets(), fun = smallvis::smallvis,
vfun = NULL, mfun = NULL, ...) {
varargs <- list(...)
if (is.null(varargs$ret_extra)) {
varargs$ret_extra <- TRUE
dataset_names <- names(datasets)
varargs_orig <- varargs
jlmelville /
Created March 14, 2020 18:00
Graph Laplacians (in Python)
import collections
import pprint
import numpy as np
import scipy as sp
import scipy.linalg
AffDeg = collections.namedtuple("AffDeg", ["W", "D"])
Lap = collections.namedtuple("Lap", ["L", "Lsym", "Lrw", "P"])
Eig = collections.namedtuple("Eig", ["vectors", "values", "lengths"])
jlmelville / spectral.R
Last active March 14, 2020 18:06
Graph Laplacians (in R)
View spectral.R
# Create the degree matrix D from an affinity/adjacency matrix
degmat <- function(X) {
# A random affinity matrix
randw <- function(n = 3) {
X <- matrix(rnorm(n * n), nrow = n)
X <- X * X
X <- t(X) + X