Skip to content

Instantly share code, notes, and snippets.

View jjerphan's full-sized avatar
☀️

Julien Jerphanion jjerphan

☀️
View GitHub Profile
@jjerphan
jjerphan / gist:44ca2e5bf102e11f47b7d3984054d72a
Last active March 29, 2017 10:19
En tête du cahier d'integration -- MT94
\documentclass[a4paper,10pt]{report}
\usepackage[francais]{babel} % Package babel pour le français
\usepackage[T1]{fontenc} % Package pour les accentuations
\usepackage[utf8]{inputenc} % Français
\usepackage{lmodern} % Pour avoir de bonnes polices en pdf
\usepackage{graphicx} % Indispensable pour les figures
\usepackage{epstopdf} % Utile pour les figures, résout une erreur
\usepackage{amsmath} % Environnement pour les maths, permet du mettre du texte dans les équations
\usepackage{textcomp}
\usepackage{geometry} % Utilisé pour les marges
@jjerphan
jjerphan / deadlock_dispatch
Last active May 22, 2019 13:53
Deadlock on joblib.Parallel when using DaskDistributedBackEnd: state of the lock and race condition in 2 different threads
[2019-05-22 13:46:14,970] [22/MainThread] [INFO] [root] Building Gridsearcher for grid: {'reg_alpha': [0.0], 'colsample_bylevel': [1.0], 'learning_rate': [0.2, 0.4, 0.8], 'max_delta_step': [0.0], 'booster': ['gbtree'], 'colsample_bytree': [1.0], 'min_child_weight': [0.0], 'subsample': [1.0], 'reg_lambda': [1.0, 3.0, 5.0], 'objective': ['reg:linear'], 'max_depth': [3, 4, 5, 6, 7, 8], 'gamma': [0.0]}
[2019-05-22 13:46:14,971] [22/MainThread] [INFO] [root] Fitting 3 folds for each of 54 candidates, totalling 162 fits
[2019-05-22 13:46:14,974] [22/MainThread] [INFO] [root] Performing GridSearch with CVInterruptWatcherThread
[2019-05-22 13:46:14,974] [22/MainThread] [INFO] [root] Performing GridSearch with Dask
[2019-05-22 13:46:14,974] [22/MainThread] [INFO] [root] Creating Dask client
[2019-05-22 13:46:15,063] [22/MainThread] [INFO] [root] Client.__init__ called from get_best_estimator
[2019-05-22 13:46:15,064] [22/MainThread] [INFO] [root] Client.start called from __init__
[2019-05-22 13:46:15,068] [22/IO loop]
@jjerphan
jjerphan / server
Created May 23, 2019 16:31
Deadlock on joblib.Parallel when using DaskDistributedBackEnd: reproducible setup (1 trial)
Showing logs of joblib-dask-deadlock-test-server-5796bf7779-6rnlc
[2019-05-23 15:58:21,433] [9/MainThread] [INFO] [root] Launching a server
[2019-05-23 15:58:21,701] [10/MainThread] [INFO] [root] Joblib 0.13.2 imported
[2019-05-23 15:58:22,291] [10/MainThread] [DEBUG] [asyncio] Using selector: EpollSelector
/usr/local/lib/python3.6/site-packages/bokeh/themes/theme.py:94: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
json = yaml.load(f)
[2019-05-23 15:58:23,109] [10/MainThread] [INFO] [root] setup_log() called
[2019-05-23 15:58:23,109] [10/MainThread] [INFO] [root] Installing debugging signal handler
[2019-05-23 15:58:23,110] [10/MainThread] [INFO] [root] Started reproducible example
[2019-05-23 15:58:23,176] [10/MainThread] [INFO] [root] Dask Scheduler: starting on port 8786
@jjerphan
jjerphan / server
Last active May 27, 2019 12:31
jjerphan/joblib_dask_deadlock: hanging on `Client._update_scheduler_info` — commit: 3f3fb9b708b5ec74607c3edb5fb32f7fac9eeea1
Showing logs of joblib-dask-deadlock-test-server-5796bf7779-4dkw2
[2019-05-27 12:17:25,463] [8/MainThread] [INFO] [root] Launching a server
[2019-05-27 12:17:25,789] [9/MainThread] [INFO] [root] Joblib 0.13.2 imported
[2019-05-27 12:17:25,939] [9/MainThread] [DEBUG] [asyncio] Using selector: EpollSelector
[2019-05-27 12:17:26,152] [9/MainThread] [INFO] [root] Distributed 1.28.0+11.g715c05c imported
[2019-05-27 12:17:26,152] [9/MainThread] [INFO] [root] Git revision: 715c05c2449c55345821f9aecb52d62f6d0d4a3b
[2019-05-27 12:17:26,153] [9/MainThread] [INFO] [joblib.dask] _dask import (logger)
[2019-05-27 12:17:26,153] [9/MainThread] [INFO] [root] _dask import (logging)
/usr/local/lib/python3.6/site-packages/bokeh/themes/theme.py:94: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
json = yaml.load(f)
@jjerphan
jjerphan / server
Created May 31, 2019 12:05
jjerphan/joblib_dask_deadlock: hanging on `Scheduler.handle_task_finished` — commit: 95a34d6bdfd3cbf8f491bab70f02a384feb1cc95
Showing logs of joblib-dask-deadlock-test-server-5796bf7779-5pxcz
[2019-05-31 11:50:20,035] [8/MainThread] [INFO] [root] Launching a server
[2019-05-31 11:50:20,771] [9/MainThread] [INFO] [root] Distributed 1.28.0+15.g6ea010b imported
[2019-05-31 11:50:20,772] [9/MainThread] [INFO] [root] Git revision: 6ea010bcf21db7445bd26286966f59a7e75ab390
/usr/local/lib/python3.6/site-packages/bokeh/themes/theme.py:94: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
json = yaml.load(f)
[2019-05-31 11:50:21,488] [9/MainThread] [INFO] [root] Joblib 0.13.2 imported
[2019-05-31 11:50:21,818] [9/MainThread] [INFO] [root] setup_log() called
[2019-05-31 11:50:21,818] [9/MainThread] [INFO] [root] Installing debugging signal handler
[2019-05-31 11:50:21,819] [9/MainThread] [INFO] [root] Started reproducible example
import numpy as np
import itertools
import time
import seaborn as sns
import gc
import pandas as pd
from matplotlib import pyplot as plt
from sklearn.calibration import CalibratedClassifierCV
from sklearn.datasets import make_classification
@jjerphan
jjerphan / trace.log
Created July 29, 2020 11:14
Trace describing ilastik#2169 — Problem when saving ("Object dtype dtype('O') has no native HDF5 equivalent") — Ilastik 1.3.3post3
ERROR 2020-07-29 11:54:39,805 slot 7664 139953135314752 setDirty called
ERROR 2020-07-29 11:54:39,806 slot 7664 139953135314752 self.stype.isConfigured
ERROR 2020-07-29 11:54:39,806 slot 7664 139953135314752 Number of Downstream_slots: 0
ERROR 2020-07-29 11:54:39,806 slot 7664 139953135314752 OpExportSlot.Input []: {_ready : True, NOTREADY : None, shape : (511, 512, 512, 3), axistags : z y x c, original_axistags : z y x c, dtype : <class 'numpy.float32'>, drange : (0.0, 1.0), has_mask : None, _dirty : False, ideal_blockshape : [0, 0, 0, 3], display_mode : 'grayscale', channel_names : ['Gaussian Smoothing (σ=0.3) [0]', 'Gaussian Smoothing (σ=0.3) [1]', 'Gaussian Smoothing (σ=0.3) [2]', 'Gaussian Smoothing (σ=0.3) [3]', 'Gaussian Smoothing (σ=0.7) [0]', 'Gaussian Smoothing (σ=0.7) [1]', 'Gaussian Smoothing (σ=0.7) [2]', 'Gaussian Smoothing (σ=0.7) [3]', 'Gaussian Smoothing (σ=1.0) [0]', 'Gaussian Smoothing (σ=1.0) [1]', 'Gaussian Smoothing (σ=1.0) [2]', 'Gaussian Smoothing (σ=1.0) [3]', 'Gaussian Smoothing (σ=
@jjerphan
jjerphan / benchmark.py
Created January 14, 2021 18:09
sklearn#18850 - benchmark
import gc
import time
import numpy as np
import pandas as pd
from scipy import linalg
"""
A simple benchmark to know the performances of setting `check_finite`
to `False` for `linalg.cholesky`
@jjerphan
jjerphan / test_system_nakagami_fit.py
Created January 21, 2021 09:24
Nakagami Loglikelihood tests: fit vs scipy.optimize.root
@pytest.mark.parametrize('nu', [1.6, 2.5, 3.9])
@pytest.mark.parametrize('loc', [25.0, 10, 35])
@pytest.mark.parametrize('scale', [13, 5, 20])
def test_fit(self, nu, loc, scale):
# Regression test for gh-13396 (21/27 cases failed previously)
# The first tuple of the parameters' values is discussed in gh-10908
N = 100
samples = stats.nakagami.rvs(size=N, nu=nu, loc=loc,
scale=scale, random_state=1337)
nu_est, loc_est, scale_est = stats.nakagami.fit(samples)
import gc
import itertools
import time
import numpy as np
import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
from sklearn.datasets import make_classification