jlmelville

June 28 2023 O frabjous day! After much fruitless occasional faffing around with cmake settings over the course of several months, new CUDA update for WSL Ubuntu means that I am now able to build the nearest neighbors library Faiss with GPU support for my 1080 laptop graphics card. Here is the setup that worked for me.

Why would you want to build your own Faiss? Well, if you are using pip to install faiss-gpu, you can't get anything more recent than version 1.7.2. Also, with recent dependency management changes with newer Pythons (particularly Python 3.11), some packages are going to be playing catch-up with getting up to parity, so it may be useful to build from source. I don't remember how bad it was trying to install faiss-gpu on Python 3.11 with pip, but it must have been fairly un-fun, because I gave up and scuttled back to the relatively welcoming environs of Python 3.10 pretty quickly after briefly dipping my

jlmelville

Ubuntu, CUDA, WSL2 and legacy keyrings

In the Ubuntu tutorial on enabling CUDA with WSL2, and specifically in part 3, you are directed to use apt-key to point Ubuntu at the correct toolkit package. On Ubuntu 22.04 this will lead to the following warning whenever you run apt-get update:

 Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.

jlmelville

why use reticulate when you can make life really difficult for yourself?

Carry out the tdf-if procedure from https://umap-learn.readthedocs.io/en/latest/sparse.html, save it in a form palatable to R:

# change this
your_data_path = "/your/data/path"

import os

jlmelville

# modified slightly from https://gist.github.com/floybix/6533090

distrib.pkgs <- library(lib.loc=R.home("library"))$results[,1]

dependencies <-
  function(pkg, dependencies = c("Depends", "Imports", "LinkingTo"),
           pl = installed.packages())
  {
    if (!(pkg %in% rownames(pl))) stop("unknown pkg ", pkg)
    fields <- pl[pkg, dependencies]

jlmelville

install.packages("devtools")
devtools::install_github("r-lib/revdepcheck")

# may need to install bioconductor
if (!require("BiocManager", quietly = TRUE)) {
  install.packages("BiocManager")
  BiocManager::install(version = "3.14")
}

library(revdepcheck)

jlmelville

import os
import pickle
from pathlib import Path


def read_data(dataset, suffix=None, dir=None, repickle=False):
    """Set `repickle=True` to rewrite pickle file if CSV has changed"""
    home = str(Path.home())
    if dir is None:
        dataset_path = os.path.join(home, "dev", "datasets")

jlmelville

whitening_matrix <- function(X, eps = 1e-8, zca = FALSE) {
  X <- scale(X, center = TRUE, scale = FALSE)
  s <- svd(X, nu = 0)

  Linvsqr <- sqrt(nrow(X) - 1) / (s$d + eps)

  W <- Linvsqr * t(s$v)
  if (zca) {
    W <- s$v %*% W
  }

jlmelville

benchmark <- function(datasets = smallvis_datasets(), fun = smallvis::smallvis, 
                      vfun = NULL, mfun = NULL, ...) {
  varargs <- list(...)
  if (is.null(varargs$ret_extra)) {
    varargs$ret_extra <- TRUE
  }

  dataset_names <- names(datasets)
  varargs_orig <- varargs

jlmelville

import collections
import pprint

import numpy as np
import scipy as sp
import scipy.linalg

AffDeg = collections.namedtuple("AffDeg", ["W", "D"])
Lap = collections.namedtuple("Lap", ["L", "Lsym", "Lrw", "P"])
Eig = collections.namedtuple("Eig", ["vectors", "values", "lengths"])

jlmelville

# Create the degree matrix D from an affinity/adjacency matrix
degmat <- function(X) {
  diag(colSums(X))
}

# A random affinity matrix
randw <- function(n = 3) {
  X <- matrix(rnorm(n * n), nrow = n)
  X <- X * X
  X <- t(X) + X