Skip to content

Instantly share code, notes, and snippets.

Avatar

Takeru Ohta sile

View GitHub Profile
@sile
sile / stars.md
Created May 5, 2022
2022/05/05時点でのリポジトリのスター数
View stars.md

starzを使って集計:

前回: 2019/04/10

$ starz sile

Total: 2087

jsone                      ★  250
@sile
sile / config.gin
Last active Apr 8, 2020
Example: gin-config with optuna
View config.gin
train.batch_size = 10
train.learning_rate = 0.1
@sile
sile / random.py
Last active Jan 27, 2020
Kurobako blog: random.py
View random.py
# A solver implementation based on Random Search algorithm.
from kurobako import problem
from kurobako import solver
import numpy as np
class RandomSolverFactory(solver.SolverFactory):
def specification(self):
return solver.SolverSpec(name='Random Search')
def create_solver(self, seed, problem):
@sile
sile / sa.py
Created Jan 27, 2020
Kurobako blog: sa.py
View sa.py
# A solver implementation based on Simulated Annealing algorithm.
from kurobako import solver
from kurobako.solver.optuna import OptunaSolverFactory
import optuna
class SimulatedAnnealingSampler(optuna.BaseSampler):
# Please refer to
# https://github.com/optuna/optuna/blob/v1.0.0/examples/samplers/simulated_annealing_sampler.py
# for the implementation.
...
@sile
sile / custom-solvers.sh
Created Jan 27, 2020
Kurobako blog: custom-solvers.sh
View custom-solvers.sh
$ kurobako solver command python random.py > solvers.json
$ kurobako solver command python sa.py >> solvers.json
$ kurobako studies --problems $(cat problems.json) --solvers $(cat solvers.json) | kurobako run > result.json
@sile
sile / report.md
Created Jan 27, 2020
Kurobako blog: report.md
View report.md

Benchmark Result Report

  • Report ID: ecb7024f149c2653a906f600d048eac2658f94bc2cfc8e50bfd7356c0fb1e29b
  • Kurobako Version: 0.1.3
  • Number of Solvers: 4
  • Number of Problems: 2
  • Metrics Precedence: best value -> AUC

Please refer to ["A Strategy for Ranking Optimizers using Multiple Criteria"][Dewancker, Ian, et al., 2016] for the ranking strategy used in this report.

@sile
sile / basic-benchmark.sh
Last active Sep 24, 2020
kurobako blog: basic-benchmark.sh
View basic-benchmark.sh
# 1. Download kurobako binary.
$ curl -L https://github.com/sile/kurobako/releases/download/0.2.6/kurobako-0.2.6.linux-amd64 -o kurobako
$ chmod +x kurobako && sudo mv kurobako /usr/local/bin/
# 2. Download the data file for HPOBench (note that the file size is about 700MB).
$ curl -OL http://ml4aad.org/wp-content/uploads/2019/01/fcnet_tabular_benchmarks.tar.gz
$ tar xf fcnet_tabular_benchmarks.tar.gz && cd fcnet_tabular_benchmarks/
# 3. Specify problems used in this benchmark.
#
View _main.md

Benchmark Result Report

  • Report ID: 4ba0c3ff34e08ddaac04387c13628701e21fd683a713b496b3f78c3d541beac9
  • Kurobako Version: 0.1.3
  • Number of Solvers: 3
  • Number of Problems: 4
  • Metrics Precedence: best value -> AUC

Please refer to ["A Strategy for Ranking Optimizers using Multiple Criteria"][Dewancker, Ian, et al., 2016] for the ranking strategy used in this report.

View _main.md

The aim of this benchmark is to compare the performances of Optuna's pruners (i.e., NopPruner, MedianPruner, SuccessiveHalvingPruner and the ongoing HyperbandPruner). All of the pruners were used by the default settings in this benchmark.

The commands to execute this benchmark.

// (1) Downloads `kurobako` (BBO benchmark tool) binary.
$ curl -L https://github.com/sile/kurobako/releases/download/0.1.3/kurobako-0.1.3.linux-amd64 -o kurobako
$ chmod +x kurobako && sudo mv kurobako /usr/local/bin/

// (2) Downloads data files of HPOBench. (notice that the total size is over 700MB)
$ curl -OL http://ml4aad.org/wp-content/uploads/2019/01/fcnet_tabular_benchmarks.tar.gz
@sile
sile / kurobako-usage.md
Last active Sep 14, 2019
kurobakoの使い方メモ
View kurobako-usage.md
$ cargo install kurobako
// or (only linux)
$ wget https://github.com/sile/kurobako/releases/download/0.0.15/kurobako-0.0.15.linux-amd64 -o kurobako && chmod +x kurobako
// or 
$ git clone git://github.com/sile/kurobako.git && cd kurobako && git checkout 0.0.14 && cargo install --path .

// 独自サンプラの場合
$ kurobako benchmark --problems (kurobako problem-suite sigopt auc) --solvers (kurobako solver command -- python3 /tmp/optuna_solver_example.py ) --budget 100 --iterations 10 | kurobako run > /tmp/sigopt-my-sampler.json