Skip to content

Instantly share code, notes, and snippets.

@yuyasugano
yuyasugano / Technical index for features with Ta-Lib
Created October 1, 2019 11:41
Technical index for features with Ta-Lib
open = pd.Series(df['open'])
high = pd.Series(df['high'])
low = pd.Series(df['low'])
close = pd.Series(df['close'])
volume = pd.Series(df['volume'])
# pct_change for new column
X['diff'] = y
# Exponential Moving Average
@yuyasugano
yuyasugano / anaconda3 of continuumio
Created October 8, 2019 12:27
conda list for anaconda3 of continuumio
_ipyw_jlab_nb_ext_conf 0.1.0 py37_0
_libgcc_mutex 0.1 main
alabaster 0.7.12 py37_0
anaconda 2019.07 py37_0
anaconda-client 1.7.2 py37_0
anaconda-navigator 1.9.7 py37_0
anaconda-project 0.8.3 py_0
asn1crypto 0.24.0 py37_0
astroid 2.2.5 py37_0
astropy 3.2.1 py37h7b6447c_0
_ipyw_jlab_nb_ext_conf 0.1.0 py37_0
_libgcc_mutex 0.1 main
alabaster 0.7.12 py37_0
anaconda 2019.07 py37_0
anaconda-client 1.7.2 py37_0
anaconda-navigator 1.9.7 py37_0
anaconda-project 0.8.3 py_0
asn1crypto 0.24.0 py37_0
astroid 2.2.5 py37_0
astropy 3.2.1 py37h7b6447c_0
@yuyasugano
yuyasugano / Univariate Statistics
Created October 30, 2019 11:32
Univariate Statistics
# Univariate Statistics
from sklearn.feature_selection import SelectPercentile
select = SelectPercentile(percentile=25)
select.fit(X_train_full, y_train_full.values.ravel())
X_train_selected = select.transform(X_train_full)
X_test_selected = select.transform(X_test_full)
mask = select.get_support()
print(mask)
plt.matshow(mask.reshape(1, -1), cmap='gray_r')
plt.xlabel("Technical Indexes")
@yuyasugano
yuyasugano / Model-based Selection
Last active October 30, 2019 12:18
Model-based Selection
# Model-based Selection
from sklearn.feature_selection import SelectFromModel
select = SelectFromModel(RandomForestClassifier(n_estimators=100, random_state=42),
threshold="1.25*mean")
select.fit(X_train_full, y_train_full.values.ravel())
X_train_model = select.transform(X_train_full)
print(X_train_model.shape)
X_test_model = select.transform(X_test_full)
mask = select.get_support()
print(mask)
@yuyasugano
yuyasugano / Recursive Feature Elimination
Created October 30, 2019 12:27
Recursive Feature Elimination
# Recursive Feature Elimination
from sklearn.feature_selection import RFE
select = RFE(RandomForestClassifier(n_estimators=100, random_state=42),
n_features_to_select=15)
select.fit(X_train_full, y_train_full.values.ravel())
X_train_rfe = select.transform(X_train_full)
X_test_rfe = select.transform(X_test_full)
mask = select.get_support()
print(mask)
plt.matshow(mask.reshape(1, -1), cmap='gray_r')
@yuyasugano
yuyasugano / Nested Cross validation GridSearch CV
Created November 3, 2019 03:20
Nested Cross validation GridSearch CV
from sklearn.datasets import load_iris
from sklearn.model_selection import ParameterGrid, StratifiedKFold
iris = load_iris()
param_grid = [{'kernel': ['rbf'],
'C': [0.01, 1],
'gamma': [0.1, 1]},
{'kernel': ['linear'],
'C': [0.01, 1]}]
print("List of parameter grids:\n{}".format(param_grid))
@yuyasugano
yuyasugano / Elasticsearch & Kibana on Docker
Created November 6, 2019 10:08
Elasticsearch & Kibana on Docker
version: '3'
services:
elasticsearch:
build: elasticsearch
container_name: elasticsearch
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms256m -Xmx256m"
ulimits:
@yuyasugano
yuyasugano / Elasticsearch Index status for the index btcjpy
Created November 6, 2019 23:25
curl -XGET "localhost:9200/btcjpy?pretty&pretty"
{
"btcjpy" : {
"aliases" : { },
"mappings" : {
"properties" : {
"close" : {
"type" : "long"
},
"timestamp" : {
"type" : "date",
@yuyasugano
yuyasugano / Elasticsearch bitbank.cc API
Created November 7, 2019 13:05
Elasticsearch bitbank.cc API data crawling every second
#!/usr/bin/python
import json
import time
import python_bitbankcc
from datetime import datetime
from elasticsearch import Elasticsearch
def get_ticker(pub):
ret = pub.get_ticker('btc_jpy')
return ret