00_odsc_ldn_2018.md

Algorithmic Trading with ML & DL

Dr. Yves J. Hilpisch | The Python Quants GmbH

Workshop at ODSC London on 20. September 2018

See http://odsc.com/london.

Links

• http://tpq.io
• http://pqp.io
• http://training.tpq.io
• http://certificate.tpq.io
• http://twitter.com/dyjh

Link to Slides

http://hilpisch.com/odsc_ldn_2018.pdf

Short Link to Gist

http://bit.ly/odsc_ldn_2018

Python

You should have installed either Anaconda or Miniconda.

If you do not have a Python installation/environment with the core scientific packages, do on the shell do:

conda create -n odsc python=3.6 jupyter pandas nomkl scikit-learn matplotlib
conda activate odsc

In any case do the following to install the fxcmpy package:

pip install --upgrade pip
pip install fxcmpy

FXCM

You should open a demo account under

https://www.fxcm.com/uk/forex-trading-demo/

The documentation for fxcmpy is found under

http://fxcmpy.tpq.io

You should generate a token in your account under Token Management.

Create your working folder where you put in all subsequently created files.

Then create a configuration file with name fxcm.cfg and the following content:

[FXCM]
log_level = error
log_file = 'fxcm.log'
access_token = YOUR_FXCM_API_TOKEN

01_fxcm_basics.ipynb

02_live_session.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" alt=\"The Python Quants\" width=\"35%\" align=\"right\" border=\"0\"><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ODSC &mdash; Live Session"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Risk Disclaimer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits. Leverage can work against you. Due to the certain restrictions imposed by the local law and regulation, German resident retail client(s) could sustain a total loss of deposited funds but are not subject to subsequent payment obligations beyond the deposited funds. Be aware and fully understand all risks associated with the market and trading. Prior to trading any products, carefully consider your financial situation and experience level. Any opinions, news, research, analyses, prices, or other information is provided as general market commentary, and does not constitute investment advice. FXCM & TPQ will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Speaker Disclaimer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The speaker is neither an employee, agent nor representative of FXCM and is therefore acting independently. The opinions given are their own, constitute general market commentary, and do not constitute the opinion or advice of FXCM or any form of personal or investment advice. FXCM assumes no responsibility for any loss or damage, including but not limited to, any loss or gain arising out of the direct or indirect use of this or any other content. Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datetime as dt\n",
    "from pylab import plt\n",
    "plt.style.use('seaborn')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fxcmpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api = fxcmpy.fxcmpy(config_file='../fxcm.cfg')  # adjust path/filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "symbol = 'EUR/USD'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw = api.get_candles(symbol, period='m1', start='2018-08-18', end='2018-08-30')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.DataFrame()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['midclose'] = (raw['bidclose'] + raw['askclose']) / 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['returns'] = np.log(data / data.shift(1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lags = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols = []\n",
    "for lag in range(1, lags + 1):\n",
    "    col = 'lag_{}'.format(lag)\n",
    "    data[col] = np.sign(data['returns'].shift(lag))\n",
    "    cols.append(col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head(8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "2 ** lags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.dropna(inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Strategy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MLPClassifier(hidden_layer_sizes=[200, 200], max_iter=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%time model.fit(data[cols], np.sign(data['returns']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.predict(data[cols])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['prediction'] = model.predict(data[cols])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['prediction'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy_score(data['prediction'], np.sign(data['returns']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Backtesting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['strategy'] = data['prediction'] * data['returns']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.exp(data[['returns', 'strategy']].sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[['returns', 'strategy']].cumsum().apply(np.exp).plot(figsize=(10, 6));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train-Test Split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automated Trading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def output(data, dataframe):\n",
    "    print('%3d | %s | %s | %6.5f, %6.5f' \n",
    "          % (len(dataframe), data['Symbol'],\n",
    "             pd.to_datetime(int(data['Updated']), unit='ms'), \n",
    "             data['Rates'][0], data['Rates'][1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.subscribe_market_data('EUR/USD', (output,))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.unsubscribe_market_data('EUR/USD') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sel = ['tradeId', 'amountK', 'currency',\n",
    "       'grossPL', 'isBuy']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "order = api.create_market_buy_order('EUR/USD', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.get_open_positions()[sel]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.close_all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.get_open_positions()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.arange(10).reshape(1, -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "position = 0\n",
    "trades = 0\n",
    "ticks = 0\n",
    "min_length = lags + 1\n",
    "def auto_trader(data, dataframe):\n",
    "    global position, trades, ticks, min_length\n",
    "    ticks += 1\n",
    "    print(ticks, end=' ')\n",
    "    resam = dataframe.resample('10s', label='right').last().ffill()\n",
    "    \n",
    "    if len(resam) > min_length:\n",
    "        min_length += 1\n",
    "        resam['mid'] = (resam['Bid'] + resam['Ask']) / 2\n",
    "        resam['returns'] = np.log(resam['mid'] / resam['mid'].shift(1))\n",
    "        features = np.sign(resam['returns'].iloc[-(lags+1):-1])\n",
    "        features = features.values.reshape(1, -1)\n",
    "        signal = model.predict(features)\n",
    "        print('\\nNEW SIGNAL: {}'.format(signal))\n",
    "        \n",
    "        if position in [0, -1]:\n",
    "            if signal == 1:\n",
    "                if position == -1:\n",
    "                    api.close_all_for_symbol(symbol)\n",
    "                api.create_market_buy_order(symbol, 1)\n",
    "                position = 1\n",
    "                print('{} | ***PLACING BUY ORDER***'.format(dt.datetime.now()))\n",
    "                \n",
    "        elif position in [0, 1]:\n",
    "            if signal == -1:\n",
    "                if position == 1:\n",
    "                    api.close_all_for_symbol(symbol)\n",
    "                api.create_market_sell_order(symbol, 1)\n",
    "                position = -1\n",
    "                print('{} | ***PLACING SELL ORDER***'.format(dt.datetime.now()))\n",
    "        \n",
    "        if ticks > 100:\n",
    "            pass\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.subscribe_market_data('EUR/USD', (auto_trader,))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.unsubscribe_market_data('EUR/USD') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" alt=\"The Python Quants\" width=\"35%\" align=\"right\" border=\"0\"><br>\n",
    "\n",
    "<a href=\"http://tpq.io\" target=\"_blank\">http://tpq.io</a> | <a href=\"http://twitter.com/dyjh\" target=\"_blank\">@dyjh</a> | <a href=\"mailto:training@tpq.io\">training@tpq.io</a>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

update.sh

rm fxcm_*
gist -u b41d0e935128a65a611e87bcf3bff665 *