Skip to content

Instantly share code, notes, and snippets.

View toshihikoyanase's full-sized avatar

Toshihiko Yanase toshihikoyanase

View GitHub Profile
@toshihikoyanase
toshihikoyanase / multinode.py
Last active February 9, 2021 12:04
Optuna PR 2303 NCCL example
"""
Optuna example that optimizes multi-layer perceptrons using PyTorch distributed.
In this example, we optimize the validation accuracy of hand-written digit recognition using
PyTorch distributed data parallel and MNIST. We optimize the neural network architecture as well
as the optimizer configuration. As it is too time consuming to use the whole MNIST dataset, we
here use a small subset of it.
You can execute this example with mpirun command as follows:
$ mpirun -n 2 python pytorch_distributed_simple.py
@toshihikoyanase
toshihikoyanase / torch_distributed_broadcast_bool.py
Created February 3, 2021 05:52
Cannot broadcast bool tensor.
import torch
import torch.distributed as dist
import torch.multiprocessing as mp
def example(rank, world_size):
dist.init_process_group("gloo", rank=rank, world_size=world_size)
if rank == 0:
z = torch.tensor([True], dtype=torch.bool)
@toshihikoyanase
toshihikoyanase / ax-knapsack-problem.ipynb
Last active January 21, 2021 04:03
Solving the multi-objective knapsack problem using Ax
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@toshihikoyanase
toshihikoyanase / constrained_domination_for_motpe.py
Last active January 19, 2021 08:02
A prototypical implementation of MOTPE with constraints
def _dominates_both_feasible(extended: np.ndarray, const_extended: np.ndarray) -> np.ndarray:
return np.logical_and(
np.logical_and(
np.all(const_extended <= 0, axis=2),
np.all(np.swapaxes(const_extended, 0, 1) <= 0, axis=2),
), np.logical_and(
np.all(extended <= np.swapaxes(extended, 0, 1), axis=2),
np.any(extended < np.swapaxes(extended, 0, 1), axis=2),
)
)
@toshihikoyanase
toshihikoyanase / README.md
Last active March 16, 2021 05:43
Migration tests for optuna/optuna#2030

Overview

This document explains how to test migrated schema of Optuna's RDBStorage. The test consists of following two parts:

  1. Test for values
  2. Test for schema

Each section describes the test instruction for each database.

@toshihikoyanase
toshihikoyanase / lightgbm_tuner_regression.py
Created October 29, 2020 01:16
Regression example of LightGBMTuner
import sklearn.datasets
from sklearn.model_selection import train_test_split
import optuna.integration.lightgbm as lgb
if __name__ == "__main__":
X, y = sklearn.datasets.load_boston(return_X_y=True)
train_x, val_x, train_y, val_y = train_test_split(X, y, test_size=0.25)
dtrain = lgb.Dataset(train_x, label=train_y)
@toshihikoyanase
toshihikoyanase / lightgbm_stepwise_tuner_simple.py
Created October 27, 2020 07:50
An example of `StepwiseLightGBMTuner`.
import numpy as np
import sklearn.datasets
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
import optuna.integration.lightgbm as lgb
if __name__ == "__main__":
data, target = sklearn.datasets.load_breast_cancer(return_X_y=True)
@toshihikoyanase
toshihikoyanase / pr1514-sql-dump.py
Created July 14, 2020 05:21
Validation code for optuna/optuna PR1514
import logging
import os
import optuna
logging.basicConfig()
logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO)
@toshihikoyanase
toshihikoyanase / pr1498.py
Created July 10, 2020 06:14
Code for reproduction for Optuna PR #1498.
from multiprocessing import Pool
import os
import sys
import optuna
def f(x, y):
return (x - 3) ** 2 + y
@toshihikoyanase
toshihikoyanase / cltpe_sampler.py
Last active February 5, 2021 13:09
A simple objective function to check constant liar.
import optuna
from optuna import distributions
from optuna.samplers import TPESampler
from optuna.study import StudyDirection
from optuna.trial import TrialState
import numpy as np
def default_gamma(x):
# type: (int) -> int