Skip to content

Instantly share code, notes, and snippets.

View toshihikoyanase's full-sized avatar

Toshihiko Yanase toshihikoyanase

View GitHub Profile
mkdir tmp
for f in *b.zip
do
echo $f
pushd tmp
unzip ../$f
popd
done
mkdir utf8
@toshihikoyanase
toshihikoyanase / main3.py
Last active August 29, 2015 14:20
A template for small scripts in python 3
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import argparse
def run():
args = parse_args()
print("Input path: {0}".format(args.input_path))
@toshihikoyanase
toshihikoyanase / twitter_streaming_filter_with_retry.py
Created June 12, 2013 22:19
Twitter Streaming APIを叩いて、Geoタグ付きのツイートを収集するPythonスクリプト
#!/usr/bin/env python
import tweepy
import datetime
import time
# See http://blog.unfindable.net/archives/4257
locationsL=[-180,-90,180,90]
class StreamListener(tweepy.StreamListener):
def __init__(self, api=None):
@toshihikoyanase
toshihikoyanase / quadratic_trial_import.py
Created February 18, 2019 04:54
An example to import existing experimental results into Optuna's storage.
"""
Optuna example that optimizes a simple quadratic function.
In this example, we demonstrate how to import existing experimental results
and continue the optimization.
We have the following two ways to execute this example:
(1) Execute this code directly.
$ python quadratic_trial_import.py
@toshihikoyanase
toshihikoyanase / quadratic_change_range.py
Last active February 18, 2019 07:21
An alternative example to import existing experimental results into Optuna's storage.
"""
Optuna example that optimizes a simple quadratic function.
In this example, we demonstrate how to import existing experimental results
and continue the optimization.
We have the following two ways to execute this example:
(1) Execute this code directly.
$ python quadratic_change_range.py

Settings: (sigopt/evalset/auc-test-suites)

  • Problems: 38
  • Metrics: best
Solver Borda Firsts
(a) optuna#tpe-faster 0 37
(b) optuna#tpe-latest 1 38

Plot of best value

@toshihikoyanase
toshihikoyanase / pruning.py
Created July 31, 2019 10:49
Pseudo-code of Pruning
import ...
def objective(trial):
...
alpha = trial.suggest_loguniform('alpha', 1e-5, 1e-1)
clf = sklearn.linear_model.SGDClassifier(alpha=alpha)
for step in range(100):
clf.partial_fit(train_x, train_y, classes=classes)
@toshihikoyanase
toshihikoyanase / nan_value_pruner_sample.py
Created August 7, 2019 05:34
An example to prune trials when they reports NaN values as intermediate values.
import math
import sklearn.datasets
import sklearn.linear_model
import sklearn.model_selection
import optuna
class NaNValuePruner(optuna.pruners.BasePruner):
@toshihikoyanase
toshihikoyanase / lightgbm_tuner_parallel.py
Created March 18, 2020 10:12
Examples for Optuna #1039 Support pruning/resume/parallelization for LightGBMTuner
"""
Optuna example that optimizes a classifier configuration for cancer dataset using LightGBM tuner.
In this example, we optimize the validation log loss of cancer detection.
You can execute this code directly.
$ python lightgbm_tuner_parallel.py [-p]
"""
import argparse