Skip to content

Instantly share code, notes, and snippets.

@nagishin

nagishin/約定履歴からOHLCV作成＆リサンプル.ipynb

Created September 18, 2018 04:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nagishin/b105049ae7983ea55280b1ec352000ef to your computer and use it in GitHub Desktop.
Save nagishin/b105049ae7983ea55280b1ec352000ef to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
約定履歴からOHLCV作成＆リサンプル.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### BitMEX API(REST)にて約定履歴取得"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[2018-09-09 20:32:31, Sell, 2876, 6355.0]\n",
"[2018-09-09 20:32:31, Sell, 10, 6355.0]\n",
"[2018-09-09 20:32:31, Sell, 20, 6355.0]\n",
"[2018-09-09 20:32:31, Sell, 60, 6355.0]\n",
"[2018-09-09 20:32:31, Sell, 636, 6354.5]\n",
"[2018-09-09 20:32:31, Sell, 100, 6354.5]\n",
"[2018-09-09 20:32:31, Sell, 100, 6354.0]\n",
"[2018-09-09 20:32:31, Sell, 31, 6354.0]\n",
"[2018-09-09 20:32:31, Sell, 100, 6354.0]\n",
"[2018-09-09 20:32:31, Sell, 16367, 6354.0]\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python3.6/site-packages/bravado_core/spec.py:271: Warning: guid format is not registered with bravado-core!\n",
" category=Warning,\n"
]
}
],
"source": [
"# coding: utf-8\n",
"from datetime import datetime\n",
"import pytz, bitmex\n",
"import pandas as pd\n",
"\n",
"now = datetime.now(pytz.utc)\n",
"# BitMEX約定履歴取得(REST API)\n",
"api = bitmex.bitmex()\n",
"trades = api.Trade.Trade_get(symbol=\"XBTUSD\", count=500, endTime=now).result()[0]\n",
"# 約定履歴リスト作成\n",
"lst_trades = [[t[\"timestamp\"], t[\"side\"], t[\"size\"], t[\"price\"]] for t in trades]\n",
"for i in range(10):\n",
" print(\"[{0[0]:%Y-%m-%d %H:%M:%S}, {0[1]}, {0[2]}, {0[3]}]\".format(lst_trades[i]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 約定履歴リストから1秒足OHLCV(DataFrame)作成\n",
"【rule】\n",
"\n",
"|rule|期間|基準|\n",
"|:---|:---|:---|\n",
"|\"AS\"|年|年初|\n",
"|\"A\"|年|年末|\n",
"|\"MS\"|月|月初|\n",
"|\"M\"|月|月末|\n",
"|\"W\"|週|日曜|\n",
"|\"D\"|日|0時|\n",
"|\"H\"|時|0分|\n",
"|\"T\"|分|0秒|\n",
"|\"S\"|秒| |"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>open</th>\n",
" <th>high</th>\n",
" <th>low</th>\n",
" <th>close</th>\n",
" <th>volume</th>\n",
" </tr>\n",
" <tr>\n",
" <th>timestamp</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2018-09-09 20:32:31+00:00</th>\n",
" <td>6355.0</td>\n",
" <td>6355.0</td>\n",
" <td>6354.0</td>\n",
" <td>6354.0</td>\n",
" <td>20300</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:32:32+00:00</th>\n",
" <td>6354.5</td>\n",
" <td>6355.5</td>\n",
" <td>6342.5</td>\n",
" <td>6348.5</td>\n",
" <td>224390</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:32:33+00:00</th>\n",
" <td>6348.0</td>\n",
" <td>6348.0</td>\n",
" <td>6342.5</td>\n",
" <td>6348.0</td>\n",
" <td>3062</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:32:34+00:00</th>\n",
" <td>6348.0</td>\n",
" <td>6348.0</td>\n",
" <td>6342.5</td>\n",
" <td>6348.0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:32:35+00:00</th>\n",
" <td>6345.0</td>\n",
" <td>6345.0</td>\n",
" <td>6345.0</td>\n",
" <td>6345.0</td>\n",
" <td>250</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" open high low close volume\n",
"timestamp \n",
"2018-09-09 20:32:31+00:00 6355.0 6355.0 6354.0 6354.0 20300\n",
"2018-09-09 20:32:32+00:00 6354.5 6355.5 6342.5 6348.5 224390\n",
"2018-09-09 20:32:33+00:00 6348.0 6348.0 6342.5 6348.0 3062\n",
"2018-09-09 20:32:34+00:00 6348.0 6348.0 6342.5 6348.0 0\n",
"2018-09-09 20:32:35+00:00 6345.0 6345.0 6345.0 6345.0 250"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# 約定履歴DataFrame作成(index設定)\n",
"df_trades = pd.DataFrame(lst_trades, columns=[\"timestamp\", \"side\", \"size\", \"price\"])\n",
"df_trades = df_trades.set_index(\"timestamp\")\n",
"\n",
"# 1秒足(\"1S\")OHLCVリサンプリング\n",
"df_ohlcv_1s = df_trades.resample(\"1S\").agg({\n",
" \"price\" : \"ohlc\",\n",
" \"size\" : \"sum\",}).ffill()\n",
"df_ohlcv_1s.columns = [\"open\", \"high\", \"low\", \"close\", \"volume\"]\n",
"display(df_ohlcv_1s.head())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1秒足OHLCVを1分足OHLCVにリサンプリング\n",
"【集計】\n",
"\n",
"|処理|内容|\n",
"|:---|:---|\n",
"|\"min\"|最小|\n",
"|\"max\"|最大|\n",
"|\"sum\"|合計|\n",
"|\"mean\"|平均|\n",
"|\"first\"|最初の値|\n",
"|\"interpolate\"|補間|"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>open</th>\n",
" <th>high</th>\n",
" <th>low</th>\n",
" <th>close</th>\n",
" <th>volume</th>\n",
" </tr>\n",
" <tr>\n",
" <th>timestamp</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2018-09-09 20:32:00+00:00</th>\n",
" <td>6355.0</td>\n",
" <td>6355.5</td>\n",
" <td>6342.5</td>\n",
" <td>6342.5</td>\n",
" <td>268002</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:33:00+00:00</th>\n",
" <td>6345.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6355.5</td>\n",
" <td>129433</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:34:00+00:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6355.5</td>\n",
" <td>6356.0</td>\n",
" <td>10107</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:35:00+00:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6353.5</td>\n",
" <td>107246</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:36:00+00:00</th>\n",
" <td>6353.5</td>\n",
" <td>6362.0</td>\n",
" <td>6353.0</td>\n",
" <td>6362.0</td>\n",
" <td>515410</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" open high low close volume\n",
"timestamp \n",
"2018-09-09 20:32:00+00:00 6355.0 6355.5 6342.5 6342.5 268002\n",
"2018-09-09 20:33:00+00:00 6345.0 6356.0 6342.5 6355.5 129433\n",
"2018-09-09 20:34:00+00:00 6356.0 6356.0 6355.5 6356.0 10107\n",
"2018-09-09 20:35:00+00:00 6356.0 6356.0 6342.5 6353.5 107246\n",
"2018-09-09 20:36:00+00:00 6353.5 6362.0 6353.0 6362.0 515410"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# 1分足(\"1T\")にリサンプリング(timestamp:デフォルト(期間開始時刻)->TradingViewタイプ)\n",
"df_ohlcv_1m = df_ohlcv_1s.resample(\"1T\").agg({\n",
" \"open\" : \"first\",\n",
" \"high\" : \"max\",\n",
" \"low\" : \"min\",\n",
" \"close\" : \"last\",\n",
" \"volume\" : \"sum\",})\n",
"display(df_ohlcv_1m.head())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1秒足OHLCVを1分足OHLCVにリサンプリング(timestampを期間終了時間にする)\n",
"【label】\n",
"\n",
"|label|内容|\n",
"|:---|:---|\n",
"|\"left\"|期間開始時刻(デフォルト)|\n",
"|\"right\"|期間終了時刻|\n",
"<br>\n",
"【closed】\n",
"\n",
"|closed|内容|\n",
"|:---|:---|\n",
"|\"left\"|基準時間が集計開始時刻(デフォルト)|\n",
"|\"right\"|基準時間が集計終了時刻|\n"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>open</th>\n",
" <th>high</th>\n",
" <th>low</th>\n",
" <th>close</th>\n",
" <th>volume</th>\n",
" </tr>\n",
" <tr>\n",
" <th>timestamp</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2018-09-09 20:33:00+00:00</th>\n",
" <td>6355.0</td>\n",
" <td>6355.5</td>\n",
" <td>6342.5</td>\n",
" <td>6342.5</td>\n",
" <td>268002</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:34:00+00:00</th>\n",
" <td>6345.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6355.5</td>\n",
" <td>129433</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:35:00+00:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6355.5</td>\n",
" <td>6356.0</td>\n",
" <td>10107</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:36:00+00:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6353.5</td>\n",
" <td>107246</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-09 20:37:00+00:00</th>\n",
" <td>6353.5</td>\n",
" <td>6362.0</td>\n",
" <td>6353.0</td>\n",
" <td>6362.0</td>\n",
" <td>515410</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" open high low close volume\n",
"timestamp \n",
"2018-09-09 20:33:00+00:00 6355.0 6355.5 6342.5 6342.5 268002\n",
"2018-09-09 20:34:00+00:00 6345.0 6356.0 6342.5 6355.5 129433\n",
"2018-09-09 20:35:00+00:00 6356.0 6356.0 6355.5 6356.0 10107\n",
"2018-09-09 20:36:00+00:00 6356.0 6356.0 6342.5 6353.5 107246\n",
"2018-09-09 20:37:00+00:00 6353.5 6362.0 6353.0 6362.0 515410"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# 1分足(\"1T\")にリサンプリング(timestamp:期間終了時刻->BitMEXタイプ)\n",
"df_ohlcv_1m = df_ohlcv_1s.resample(\"1T\", label=\"right\").agg({\n",
" \"open\" : \"first\",\n",
" \"high\" : \"max\",\n",
" \"low\" : \"min\",\n",
" \"close\" : \"last\",\n",
" \"volume\" : \"sum\",})\n",
"display(df_ohlcv_1m.head())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### timestampのタイムゾーン変換(UTC->JST)"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>open</th>\n",
" <th>high</th>\n",
" <th>low</th>\n",
" <th>close</th>\n",
" <th>volume</th>\n",
" </tr>\n",
" <tr>\n",
" <th>timestamp</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2018-09-10 05:33:00+09:00</th>\n",
" <td>6355.0</td>\n",
" <td>6355.5</td>\n",
" <td>6342.5</td>\n",
" <td>6342.5</td>\n",
" <td>268002</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-10 05:34:00+09:00</th>\n",
" <td>6345.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6355.5</td>\n",
" <td>129433</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-10 05:35:00+09:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6355.5</td>\n",
" <td>6356.0</td>\n",
" <td>10107</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-10 05:36:00+09:00</th>\n",
" <td>6356.0</td>\n",
" <td>6356.0</td>\n",
" <td>6342.5</td>\n",
" <td>6353.5</td>\n",
" <td>107246</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-09-10 05:37:00+09:00</th>\n",
" <td>6353.5</td>\n",
" <td>6362.0</td>\n",
" <td>6353.0</td>\n",
" <td>6362.0</td>\n",
" <td>515410</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" open high low close volume\n",
"timestamp \n",
"2018-09-10 05:33:00+09:00 6355.0 6355.5 6342.5 6342.5 268002\n",
"2018-09-10 05:34:00+09:00 6345.0 6356.0 6342.5 6355.5 129433\n",
"2018-09-10 05:35:00+09:00 6356.0 6356.0 6355.5 6356.0 10107\n",
"2018-09-10 05:36:00+09:00 6356.0 6356.0 6342.5 6353.5 107246\n",
"2018-09-10 05:37:00+09:00 6353.5 6362.0 6353.0 6362.0 515410"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# タイムゾーンを日本時間(JST)に変換\n",
"df_ohlcv_1m.index = df_ohlcv_1m.index.tz_convert(\"Asia/Tokyo\")\n",
"display(df_ohlcv_1m.head())"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.2"
},
"toc": {
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment