Skip to content

Instantly share code, notes, and snippets.

@stsievert
stsievert / Example.ipynb
Last active June 5, 2018 19:37
Hyperband example
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@stsievert
stsievert / echo_client.py
Last active July 5, 2018 15:28
PyTorch and dask.distributed latency
from __future__ import print_function
import socket
import sys
import time
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 10000)
@stsievert
stsievert / prototype.py
Last active June 17, 2018 22:16
Adaptive parameter search
import numpy as np
from sklearn.model_selection import cross_validate
from sklearn.linear_model import SGDClassifier
from sklearn.datasets import make_classification
import scipy.stats as stats
from sklearn.base import clone
from pprint import pprint
import sklearn.model_selection
@stsievert
stsievert / dask-compute.pytb
Last active June 18, 2018 23:27
Hyperband traceback with dask.compute
distributed.core - ERROR - "('array-ce35610051c66c361205cb57db0700ff', 89, 7)"
Traceback (most recent call last):
File "/Users/ssievert/anaconda3/envs/dask-master/lib/python3.6/site-packages/distributed/core.py", line 375, in handle_stream
handler(**merge(extra, msg))
File "/Users/ssievert/anaconda3/envs/dask-master/lib/python3.6/site-packages/distributed/scheduler.py", line 1355, in update_graph
ts = self.tasks[key]
KeyError: "('array-ce35610051c66c361205cb57db0700ff', 89, 7)"
distributed.core - ERROR - "('array-ce35610051c66c361205cb57db0700ff', 89, 7)"
Traceback (most recent call last):
File "/Users/ssievert/anaconda3/envs/dask-master/lib/python3.6/site-packages/distributed/core.py", line 375, in handle_stream
@stsievert
stsievert / hyperband.py
Last active June 20, 2018 22:30
Hyperband sketch
import numpy as np
from distributed import Client, as_completed
from sklearn.base import clone
from sklearn.svm import SVC
import dask.array as da
from time import sleep
from sklearn.base import BaseEstimator
def set_and_clone(x):
@stsievert
stsievert / Hyperband-example.ipynb
Last active October 23, 2020 11:59
Hyperband demo with sklearn
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@stsievert
stsievert / xgboost.ipynb
Last active June 30, 2018 01:08
XGBoost + Dask example
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@stsievert
stsievert / Criteo.ipynb
Last active January 27, 2020 14:16
Criteo dataset example
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@stsievert
stsievert / sparse_einsum_fail.py
Created July 20, 2018 21:02
Sparse and opt-einsum fail
import numpy as np
from opt_einsum import contract, helpers, contract_expression
import sparse
import pytest
tests = [
'ab,bc->ca',
'abc,bcd,dea',
'abc,def->fedcba',