rth

Benchmarking matrix multiplications with numpy in Pyodide 0.17.0dev0

This gist includes benchmarks for matrix multiplication (DGEMM) using numpy in Pyodide.

Currently Pyodide includes Netlib (reference) BLAS via CLAPACK, and it would useful to replace if by a high performance BLAS such as BLIS in the future (pyodide#227). Related to benchmarks done by @ethanhs for Blis with WebAssembly https://twitter.com/ethanhs/status/1381500482858803200

rth

rth

import numpy as np
#from scipy.special import chdtrc
from scipy.sparse import spdiags

from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.preprocessing import LabelBinarizer


def _chisquare(f_obs, f_exp, reduce):
    """Replacement for scipy.stats.chisquare with custom reduction.

rth

rth

/home/rth/src/scikit-learn/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/home/rth/src/scikit-learn/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
/home/rth/src/scikit-learn/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20
  DeprecationWarning)
/home/rth/src/scikit-learn/sklearn/utils/validation.py:661: DataConversio

rth

Numpy Blas and AVX detection

Comment to https://twitter.com/honnibal/status/918427934700032000

Numpy with MKL

For numpy with MKL installed by conda the proposed solution doesn't appear to work,

rth

"""
This script aims to benchmark different parallelization options for pairwise metrics in scikit-learn.
The results can be found in https://github.com/scikit-learn/scikit-learn/issues/8216

The environement is setup with,
  
  conda create -n sklearn-env scikit-learn==0.18.1 jupyter python==3.5
  
and this benchmark should be run with,
  

rth

from sklearn.metrics import pairwise_distances
from sklearn.externals.joblib import Parallel, delayed, dump, load
import numpy as np
import tempfile
import os
from sklearn.utils import gen_even_slices
import shutil

os.environ['TMPDIR'] = '/dev/shm/'
np.random.seed(99999)

rth

from math import sqrt, acos
import numpy as np

def make_test_arrays(N, norm=True):  
    X = np.random.rand(N)
    Y = np.random.rand(N)
    if norm:
        X /= np.linalg.norm(X)
        Y /= np.linalg.norm(Y)
    return X, Y

rth