Skip to content

Instantly share code, notes, and snippets.

@timelyportfolio
Created October 25, 2013 14:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save timelyportfolio/7156009 to your computer and use it in GitHub Desktop.
Save timelyportfolio/7156009 to your computer and use it in GitHub Desktop.
Backtest Overfitting | Translated in R
<!DOCTYPE html>
<html>
<head>
<title>Backtest Overfitting | Translated in R</title>
<meta charset="utf-8">
<meta name="description" content="Backtest Overfitting | Translated in R">
<meta name="author" content="TimelyPortfolio">
<meta name="generator" content="slidify" />
<meta name="apple-mobile-web-app-capable" content="yes">
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<link rel="stylesheet" href="http://slidifylibraries2.googlecode.com/git/inst/libraries/frameworks/io2012/css/default.css" media="all" >
<link rel="stylesheet" href="http://slidifylibraries2.googlecode.com/git/inst/libraries/frameworks/io2012/css/phone.css"
media="only screen and (max-device-width: 480px)" >
<link rel="stylesheet" href="http://slidifylibraries2.googlecode.com/git/inst/libraries/frameworks/io2012/css/slidify.css" >
<link rel="stylesheet" href="http://slidifylibraries2.googlecode.com/git/inst/libraries/highlighters/highlight.js/css/solarized_light.css" />
<base target="_blank"> <!-- This amazingness opens all links in a new tab. --> <link rel=stylesheet href="http://slidifylibraries2.googlecode.com/git/inst/libraries/widgets/bootstrap/css/bootstrap.css"></link>
<link rel=stylesheet href="http://fonts.googleapis.com/css?family=Raleway:300"></link>
<link rel=stylesheet href="http://fonts.googleapis.com/css?family=Oxygen"></link>
<!-- Grab CDN jQuery, fall back to local if offline -->
<script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-1.7.min.js"></script>
<script>window.jQuery || document.write('<script src="http://slidifylibraries2.googlecode.com/git/inst/libraries/widgets/quiz/js/jquery.js"><\/script>')</script>
<script data-main="http://slidifylibraries2.googlecode.com/git/inst/libraries/frameworks/io2012/js/slides"
src="http://slidifylibraries2.googlecode.com/git/inst/libraries/frameworks/io2012/js/require-1.0.8.min.js">
</script>
</head>
<body style="opacity: 0">
<slides class="layout-widescreen">
<!-- LOGO SLIDE -->
<slide class="title-slide segue nobackground">
<hgroup class="auto-fadein">
<h1>Backtest Overfitting | Translated in R</h1>
<h2></h2>
<p>TimelyPortfolio<br/></p>
</hgroup>
<a href="https://github.com/timelyportfolio/research_lopezdePrado/zipball/gh-pages" class="example">
Download
</a>
<article></article>
<footer class = 'license'>
<a href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>
<img width = '80px' src = 'http://mirrors.creativecommons.org/presskit/buttons/88x31/png/by-nc-sa.png'>
</a>
</footer>
</slide>
<!-- SLIDES -->
<slide class="" id="slide-1" style="background:;">
<article data-timings="10">
<style>
iframe{
height:450px;
width:900px;
margin:auto auto;
}
body{
font-family: 'Oxygen', sans-serif;
}
h1,h2,h3,h4 {
font-family: 'Raleway', sans-serif;
}
h3 {
background-color: #D4DAEC;
text-indent: 100px;
}
h4 {
text-indent: 100px;
}
iframe {height: 300px; width: 900px}
</style>
<h3>Original Paper</h3>
<p><br>
<address>
<strong style="lineheight:40px;">Pseudo-Mathematics and Financial Charlatanism:</strong><p style="lineheight:40px;">The Effects of Backtest Overfitting on Out-of-Sample Performance</p></p>
<p class="muted" style="line-height:26px;">Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim<br>
October 7, 2013<br>
Available at SSRN: <a href="http://ssrn.com/abstract=2308659">http://ssrn.com/abstract=2308659</a>
</p>
<p></address></p>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-2" style="background:;">
<hgroup>
<h3>Constants - R Equivalents</h3>
</hgroup>
<article data-timings="">
<pre><code class="r"># these are the constants referenced
# Euler–Mascheroni&#39;s constant
-digamma(1)
</code></pre>
<pre><code>## [1] 0.5772
</code></pre>
<pre><code class="r"># euler constant
exp(1)
</code></pre>
<pre><code>## [1] 2.718
</code></pre>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-3" style="background:;">
<hgroup>
<h3>Expected Maximum Sharpe \(E[max_N]\)</h3>
</hgroup>
<article data-timings="">
<p>\[
\begin{aligned}
(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right] \end{aligned}
\]</p>
<h4>R Translation</h4>
<pre><code class="r">### Eq. 4
emax &lt;- function(N) {
( ( 1 + digamma(1) ) * qnorm( 1 - 1/N ) ) +
(-digamma(1) * qnorm( 1 - (1/N) * exp(-1)))
}
emax( N = 10 ) # should be about = 1.57 to match paper
</code></pre>
<pre><code>## [1] 1.575
</code></pre>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-4" style="background:;">
<hgroup>
<h3>Plot \(E[max_N]\)</h3>
</hgroup>
<article data-timings="">
<pre><code class="r">#implement upper bound
upbound &lt;- function(N) {sqrt(2*log(N))}
#make a ugly plot for reasonableness check
curve(upbound, from = 1, to = 1000, col = &quot;red&quot;, lty = 2)
curve(emax, from = 1, to = 1000, add=TRUE)
grid()
</code></pre>
<p><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAxgAAAFoCAMAAADjHq2+AAAAflBMVEX9/v0AAAAAADkAAGUAOTkAOY8AZrU5AAA5ADk5AGU5OWU5OY85j9plAABlADllAGVlOQBlOY9lZmVlZrVltf2POQCPOTmPOWWPZgCPtY+P29qP2/21ZgC1ZmW1/rW1/v3T09Pajzna/tra/v39tWX924/9/rX9/tr9/v3/AAClybC8AAAAKnRSTlP/////////////////////////////////////////////////////AP+MjuKjAAAACXBIWXMAAAsSAAALEgHS3X78AAANN0lEQVR4nO3dDXca1xWF4WA3VlxLUdNK6YdoBbEA6f//wTKA5EjaoLnMnTl75rzPWtRt7MAusHXPncHDT48A3vkpOgDgiGIAAsUABIoBCBQDECgGIFAMQKAYgEAxAIFiAALFAASKAQgUAxAoBiBQDECgGIBAMQCBYgACxQAEigEIFAMQKAYgUAxAoBiAQDEAgWIAAsUABIoBCBQDECgGIFAMQKAYgEAxAIFiAALFAASKAQgUAxAoBiBQDECgGIBAMQCBYgACxQAEigEIFAMQKAYgUAxAoBiAQDEAgWIAAsUABIoBCBQDECgGIFAMQKAYgEAxAIFiAEKXYsyA8eqxGB3+XSBWRDEWHe60DhJYRHBOQDGyJjCI4JyAUQoQKAYgMEplTWAQwTkBxciawCCCcwJGKaTw1BC/HkMxMHL6Dd+2AMcwSmVNYBDhdIJ2b/iz3vUtElCMrAniIzw1Cfp5w7flVQxk0O+P+t5RDJyn9lBvhlEqa4K2EXp848c/CV6jlPPzkSfBjwhhP/njnwSvYiDGKGf9KBRjqihAJ4xSI07QbQaayJPQUwKKMZoEtZeAUT4JgyVglDI0yeOfI0MxbLAZcMIoFWBbgEX8ipD+ZTiZgGIM5tWKEP8cGERwTsAo1SP2CONFMSrixMF0MEp1dlYR4p8DgwjOCSjGGSqMSPHPgUEE5wSMUq0xImVCMT7E5jkjRilhgKNJ8c+BQQTnBBTjTwYcleKfA4MIzgkYpdg7QEhcDPYOOC7lKPW0CC9E+HPgEGH4BG+/NslrlAp7RV5GpozvCcMIFROc+f1hXsUYHHuI0ev2jXnnPODp355CMSiEv45fFdlHotO/PepR6vjuelJTxBgi6Hf7Iu59f+A1SvX9inx8uCn+bRmfYNAJv/cEZ/IqRo8Ym3pkNOj0b1LFoBYVpHr7HzeVUaqsE85r+JARgt/+8U+C1yhV8/k470is8ytS35EFYBG+Bji/DOMepZid3sk+AtUy1mLwMafH8Elo0sY5SmW8OmX1EozxSRguQedirC5ml/PZ7PN9+7vu9HxUWSqcX5HGIEuB+5MQm6BrMTa3N4/zbStW374X3vVZpjtAMQ956VqM9fX948Pl/tfnu3w+6rGtIzd52z43sze38EzcXt1qrBiNkhVj8dGdahVXizMTdPFmRQhI8FZ8BOcEnfcY66umGcu+9xh1j0IN84qcmo7i3xMGEZwTjOBw7Zj2FewUpsK+GPa1oAuT5H4eo59adFzDK3QhfoowiOCcwL0Y/Tg3Qb11If45MIjgnMB5lLIYopiUcvItRvjmgi5kZjtK9VqLYwmGWx7ipwiDCM4JbIvRq3cJBh+V4p8DgwjOCSxHqSGHKAYmKIbFGGhzwX4aJ/iNUv3XYn9Bo94f5qT4KcIggnMCv2L0yOgjfPEJDCI4JzAcpfrA2IQyXsWoP0axk8BZnEapurvuU41wXsMTRXBOYFSMerX4cJFwfkUSRXBO4DNK1eoFgxMq8ClGBWwnUIvRKNXloQq32M5reKIIzglMitFhjjpnlXB+RRJFcE7gMUqd2wtGJ/RkxMWgFOiPxSh1Ri+6tcJ5DU8UwTnBKIvRea1wfkUSRXBO4DFKldwrExQGMK5iUAoMxGKUanl/9VrhvIYniuCcIL4Y7TYYddcK51ckUQTnBOGjVKteMEFhYGMoBrXA4KJHqQ970ct+23kNTxTBOUG7YjzsPqN3U/agNYrR02Lh/IokiuCcoE0xHmZfml82t2XV6P6eZoZClBbFWP/28i1i/33/vUln3/UAdwCcK3qPceJf77EXzmt4ogjOCVoVY79mbP5Zsl60KsbxHUa/U5TzK5IognOCdivG6uLL4/zTXdmDtnhrHysGmwsEaztKPZQek2pTjKO9KH0ooLKIFeNl/TpSjP574byGJ4rgnCB0jyGLMcQY5fyKJIrgnKDN4dp/vPzPAQ7XMkbBgdsJPnoBC9EfCSmLU4vzGp4ognOCwBN8Yocx1Hrh/IokiuCcIPBj5++LwRwFF20231eHK2B+rnRU6uBtMTirBx/tVoz55fY/ll8q3bUepYashfManiiCc4J25zGu71/+s72yYgy6XDi/IokiOCdoVYzN7W7F+Pn76T9cdtcd/jDQt3aj1OZ2u8UonKSK3uv0Al5M/j7GwMVwXsMTRXBO0LkYq4tZ8+lCsQEp2GMMvWA4vyKJIjgnKDnzrQ7Xbm5vdluQkmIcBBYD+EC7o1JXRz8Nsi/E/MufinE46zFbbOt44vb047/PTv05btwCbgWHa6Vmxdha/uVr6Sj19PEf7M1i8Ef0S2AQwTlBwQk+bX21+73l+zmLYlgnMIjgnKDlKNXHR0JK/xwwnPBr11IMOIo/jxHRC+c1PFEE5wQRo9TrPQbFSBvBOUHBirE8vgM/566fWv0pIERBMap9unaPYsBYQTEe+hilYnrhvIYniuCcoGSPUfdiCBQjfQTnBNGHa5mkYCm4GPQCngr+olLhX+BrdR4jqBjOa3iiCM4JIv5q6489RtSC4fyKJIrgnCDiYgh7kcUAPhB3MYQnegFfcXuMyGI4r+GJIjgniPsQIcVIH8E5QeThWiYp2Iq8rhTFgK24w7WRvXBewxNFcE4Qdu3aJ4qRPoJzgoirne8c+ypjwEHY92NQDDgLO1wbWgznNTxRBOcEFCNrAoMIzglMvgYA8FJyuLbq5htwFvhVY4FIYBHBOUG7UWq3WNT9ckr2GNHiIzgnKLkYQuEBWw7XYrzCPkRIMeCMw7VZExhEcE4Qdu1aihEtPoJzgrhr1wLG4q5dCxiLu3ZtJBJYRHBOEHbtWvYY0eIjOCfgcC0gRBWDXsBa1HmM2GI4r+GJIjgnoBhZExhEcE4Q/f0YgCWKAQjx3/MdgQQWEZwTsMfImsAggnMCDtcCAsUABEaprAkMIjgnoBhZExhEcE7A4VpAoBiAwHmMrAkMIjgnYI+RNYFBBOcEHK4FBIoBCF2LIS5SePgHs8V2nTK9LQwycNu9ENEZjt06rxj7K6Er7DGsExhEcE7QfZRa/3p31l0DxjiPAQicx8iawCCCcwLOY2RNYBDBOQGHawEhqBj0At6CRqngYjiv4YkiOCegGFkTGERwTsDhWkCgGIDAeYysCQwiOCdgj5E1gUEE5wQcrgUEigEIjFJZExhEcE5AMbImMIjgnIDDtYBAMQCB8xhZExhEcE7AHiNrAoMIzgk4XAsIFAMQGKWyJjCI4JyAYmRNYBDBOQGHawGBYgAC5zGyJjCI4JwgphjRB6WcX5FEEZwTxIxS0cUAPkAxAIFRKmsCgwjOCShG1gQGEZwTcLgWECgGIHAeI2sCgwjOCdhjZE1gEME5AYdrAYFiAAKjVNYEBhGcE1CMrAkMIjgn4HAtIFAMQOA8RtYEBhGcE7DHyJrAIIJzAg7XAkJIMegF3IWMUuHFcF7DE0VwTkAxsiYwiOCcgMO1gEAxAIHzGFkTGERwTsAeI2sCgwjOCThcCwidi7G6mDU+3xfcNcWAu67F2Nze7H59+Pl767tmlHJIYBDBOUHXYqyv71/92tzlwWL7qPr2dOSfD3ZbBD8+t91tYZDh2C1ixQDsdd5jrK/K9xiAO85jZE1gEME5AcXImsAggnMCPhICCBQDEBilsiYwiOCcoM9iHLM4+jtDIYFFBOsE/RXjeGP6uFMSjC/CiBNQjMkmMIgw4gQUY7IJDCKMOAHFmGwCgwgjTkAxJpvAIMKIE1CMySYwiDDiBBRjsgkMIow4QS/FAMaOYgACxQAEigEIFAMQKAYgUAxAoBiAQDEAgWIAQv1irK9m76/NNozmMrs3LwliguwuUBeZYHM7+3QXGmH7MjRXIYtLsPr67uGLU1QvRvPGWH6pfa+trH+9e1z9cndIEBRkua1maIL5TXPVyMAIzcuwjEzw0PTy9cOXp6hejOYat7vGDu+h+X8+vzkkiAmy+uvfbh4jExyuMRwYYfXte/PwYQnmn/69fbzXD1+eonoxdk/L9mdGkO1DHxKEBNn8/p/tz6bIBKtv/2pGqcAIhxUjMEFTgdcPX56iejGaiz/HFWNze/mcICTI8rJZtCMTrC52xYyMsJ/nAxM0xXj98OUpprVirK8uHyN/WG4fcxO+YnT7SVkhwS93jw+f71kxXgvcY+x+WoaO18vdBYsuQ/cYv+3eAoERDj+dI3c5jnuMZpgJOiq178VzgqAgzYoRmmB+s1+4wiIcVozIBF/v37wG5SmmdB5j//P6Jvl5jO2Dxp5FeHyYBZ9JsTyPAUwBxQAEigEIFAMQKAYgUAxAoBiAQDEAgWIAAsUABIoBCBQDECgGIFAMQKAYgEAxAIFiAALFAASKAQgUAxAoBiBQDECgGJ7mYVfnwg7F8LS+/t91zOUcsUMxTC1nl9ERUqMYppoLXSIOxTA1/ztbjEgUw9Pq2x+/s2QEohiWnr9+BlEoBiBQDECgGIBAMQCBYgACxQAEigEIFAMQKAYgUAxAoBiAQDEAgWIAAsUABIoBCBQDECgGIFAMQPg/MU3zAT6PWZgAAAAASUVORK5CYII=" alt="plot of chunk unnamed-chunk-4"> </p>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-5" style="background:;">
<hgroup>
<h3>rChart \(E[max_N]\)</h3>
</hgroup>
<article data-timings="">
<pre><code class="r">require(rCharts)
df &lt;- data.frame(
list(x=c(1,2:1000), y=c(0,emax(2:1000))))
d1 &lt;- dPlot( y ~ x, groups = &quot;x&quot;, data = df, type = &quot;line&quot;, height = 270, width = 800)
d1$xAxis(type = &quot;addMeasureAxis&quot;,orderBy = &quot;x&quot;,outputFormat = &quot;,0.0f&quot;)
d1$yAxis( outputFormat = &quot;.2f&quot;)
d1
</code></pre>
<iframe src=assets/fig/unnamed-chunk-5.html seamless></iframe>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-6" style="background:;">
<hgroup>
<h3>Check My Math with <em>Eq. 6</em></h3>
</hgroup>
<article data-timings="">
<pre><code class="r">### Try next example for Eq. 6
# if y = 5
# so solve for annualized Sharpe of 1
# says no more than 45 N should be tried
# first just do this to make sure I understand
N = 45
y = 5
emax( N ) * y^-0.5 #seems like on the right path
</code></pre>
<pre><code>## [1] 0.9998
</code></pre>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-7" style="background:;">
<hgroup>
<h3>Minimum Backtest Length \(MinBTL\)</h3>
</hgroup>
<article data-timings="">
<p>\[
\begin{aligned}
\left(\frac{(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right]}{\overline{E[max_N]}}\right)^2 \end{aligned}
\]</p>
<h4> R Translation </h4>
<pre><code class="r">#use emax from earlier for numerator
minBTL &lt;- function( N, eMaxSharpe = 1 ) {
(emax(N) / eMaxSharpe) ^ 2
}
#then this should equal 5 if correct
minBTL( N = 45, eMaxSharpe = 1 )
</code></pre>
<pre><code>## [1] 4.998
</code></pre>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-8" style="background:;">
<hgroup>
<h3>Plot \(MinBTL\)</h3>
</hgroup>
<article data-timings="">
<pre><code class="r">#make ugly plot for a reasonableness check
curve( minBTL, from = 1, to = 1000)
</code></pre>
<p><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAxgAAAFoCAMAAADjHq2+AAAAeFBMVEX9/v0AAAAAADkAAGUAOY8AZrU5AAA5ADk5AGU5OWU5OY85j9plAABlADllAGVlOQBlOY9lZmVlZrVltf2POQCPOTmPOWWPZgCPtY+P29qP2/21ZgC1ZmW1/rW1/v3ajzna24/a/tra/v39tWX924/9/rX9/tr9/v0jL9iuAAAAKHRSTlP///////////////////////////////////////////////////8AvqouGAAAAAlwSFlzAAALEgAACxIB0t1+/AAACZpJREFUeJzt3WtbG9cVgNEoNsQ1mKYF9wINtNbt///DSgKcKGzkGc3l7Dmz1heePLJH26BXZ45GET9tgTd+Kj0AZCQMCAgDAsKAgDAgIAwICAMCwoCAMCAgDAgIAwLCgIAwICAMCAgDAsKAgDAgIAwICAMCwoCAMCAgDAgIAwLCgIAwICAMCAgDAsKAgDAgIAwICAMCwoCAMCAgDAgIAwLCgIAwICAMCAgDAsKAgDAgIAwICAMCwoCAMCAgDAgIAwLCgIAwICAMCAgDAsKAgDAgIAwICAMCwoCAMCAgDAh0CWMB0zVgGB3+LpQlDAgIAwLCgIAwICAMCAgDAsJgfhpcrRAGs9Di4t3znz99szCYrLYtHP/l0zcLg+k5s4XjY5y+WRhMQqfVITzg6ZuFQVa9t3B89NM3C4NEhm3h+K5O3ywMyhqxheP7PX2zMBhdqRaOhzh9szAYUdkWjnQOY3V5+Md8eGx7aNgmWR4CXcPY3N0evi4/fmt5aOYqawtHuoaxvnk8+tr80MxO8haOWDEY0iRWh0jnPcb62h6DwPRaOOJVKXoz2eUhMEAYtXxraKzCH7gVg7PUtDpEhEFL1bZwpPPLtdcvTxpvd9/Vf+/mpt7lIdB5xdjcXZ13aCag9hOm93U/lVp/uT/r0GQ2xxaO2GPwYr6rQ0QYzGQ73Y4w5szy8C5hzJMifkAYM2ORaEYY9bOrPoMwaqaFswmjSpaHroRRGUX0Qxi1sEj0ShjTp4gBCGPCLBLDEcYkKWJowpgWi8RIhDEVihiVMNKzSJQgjMQUUY4wMrJIFCeMXBSRhDCSsEjkIoziFJGRMEqSRFrCKEYTmQmjBCtFesIYlS32VAhjJIqYFmEMTxITJIyBaWKahDEcK8WECWMAttjTJ4yeKaIOwuiRKOohjH44d6qMMLrTRIWE0Y0oKiWMszl7qpkwzqKJ2gmjNVHMgTDaEcVMCKMFVcyHMJqx054ZYfyQJuZIGKdpYqaEcYIq5ksY73D+NG+dw1hdLi6eFosPj20PnZkoZq9rGJuv99uni10fn7+1PHRaqqB7GOubx+3T1fPX10O+6j7d+CY6Nn2zYvxuqjEzgD72GFc17DFEwR95VWpPFfyJMFRBYOZh2FYQm3MYouBd8w1DFZww1zBkwUnNwlgeLtjd9nrokmTBDzQJY7m42H/Z3LVLI+mDz36bBhqEsf71+0Xt/7y9jHf2oYsQBc3Mao+hCppqFMbzmrH5R5v1Il8YsqC5ZivG6vJi+/Dzfa+HHpksaKPpqdSy7WtSmcKw36at+lcMUXCG2vcYquAsTV6u/fv3/5zay7Wy4Ew1X+CTBWer9y0hsqCDWi/wyYJOWoTxhw8C6ePQQ5IFHdUYhizorMIwZEF31YVhuaAPlYUhC/rR5ALf9ctHbgYfqtbh0AOQBX2p6OVaWdCfJitGy1OohofumSzoUyVhyIJ+1RGGLOhZDZtvywW9q2DFkAX9m3wYlguGMPEwZMEwJn0dQxYMpUkY+99YvGy99x4+DFkwmAZhbO5ut0+7KpYf3/z+yS6H7spywYAa7jH2v5Q115sIZcGQJhqG5YJhTTMMWTCwKV75tlwwuAm+XCsLhje9MHTBCNp84FqGUymnUYyi2Yc6X7f+HQA/PvRZZME4moVx1rulBngQ64KRNDuVerjq/9ApjgixhqdSKfYYumA0E3pVSheMp3MYq8vbzd1iEbzBsOcHsi4YUaO3hPz2/qnU/q23D7fbw3tG2h26JV0wpq4rxvrmcfP1/uiFq8Wr7tP9TheMqvMFvt1ysbza/YmLtoduRReMq9mrUl9O/Cbjh0M0b7vo8cHscjdjm8IFPlkwumanUk8lL/DpgvE1DKPgBT5dUED3Pca5hx71KNBO9j2GLigi+R5DF5SR/E2EwqCM3G8i1AWFpA5DF5SSOQxdUEziMHRBOXnD0AUFpQ1DF5SUNQxdUJQwIJA0DF1QVs4wdEFhKcPQBaVlDEMXFCcMCCQMQxeUJwwI5AtDFySQLgxdkIEwIJAtDF2QgjAgkCwMXZBDrjB0QRLCgECqMHRBFsKAQKYwdEEawoBAojB0QR7CgECeMHRBIsKAQJowdEEmwoBAljB0QSrCgECSMHRBLsKAgDAgkCMMXZCMMCDQTxirT4+tD33mH4UxdA1jfb149uFNGs0f7bogm84rxvp6l8TRirF41dcQMLoeTqXW1x//2+1UShhk08seY3X59kSqxcNdF6ST4VUpYZCOMCCQIAxdkI8wICAMCJQPQxckJAwICAMCxcPQBRkJAwLCgIAwIFA6DF2QkjAgIAwICAMChcPQBTkJAwLCgIAwIFA2DF2QlDAgIAwICAMCRcPQBVkJAwLCgIAwIFAyDF2QljAgIAwICAMCBcPQBXkJAwLCgEDp/+cbUhIGBIQBAWFAQBgQEAYEhAGBIcOA6RoujPeLGeKgJpjeCBOeQBjVTpBghAlPIIxqJ0gwwoQnEEa1EyQYYcITCKPaCRKMMOEJhFHtBAlGmPAEwqh2ggQjTHiCQcKAqRMGBIQBAWFAQBgQEAYEhAEBYUBAGBAQBgT6D2N9vfj4rfejNrK6XCxuv09QZpDNXeEJNneLn++LjrD7MXx4LDnB6tObu289Re9h7B8YTxd9H7WR9Zf77eqX+5cJCg3ytEuz6AQPt9vlx28FR9j/GJ5KTrDcd3l89+2n6D2M9c3jc7HjW+7/5Q+3LxOUGWT1l7/ebktOsL/T7bbkCKvP3/Z3X2yCh5//tbu/47tvP0XvYRy+LbvnjEJ2d/0yQZFBNl//vXtuKjnB6vM/96dSBUd4WTEKTrBP4Pju20/Rexi7ZbxgGJu7q9cJigzydLVftEtOsLo8hFlyhOfz+YIT7MM4vvv2U9S1Yqyvr7Ylnyx397kpvmJ0e6bsYYJf7rfLD49WjGMF9xiHZ8uip9dPhw8suiq6x/j18BAoOMLLs3PJXU7GPcb+ZKbQq1LPXbxOUGiQ/YpRdIKH2+eFq9gILytGyQk+Pf7pZ9B+ipquYzw/X9/O/DrG7k7LXkXYLheFr6SkvI4BNRAGBIQBAWFAQBgQEAYEhAEBYUBAGBAQBgSEAQFhQEAYEBAGBIQBAWFAQBgQEAYEhAEBYUBAGBAQBgSEkdNDsU/n4kAYOa1vfrsp83GOHAgjqafFVekRZk0YSe0/6JJyhJHUw99sMUoSRk6rz//7askoSBgpvf76GUoRBgSEAQFhQEAYEBAGBIQBAWFAQBgQEAYEhAEBYUBAGBAQBgSEAQFhQEAYEBAGBIQBgf8DYJLZ1nNOUYcAAAAASUVORK5CYII=" alt="plot of chunk unnamed-chunk-8"> </p>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-9" style="background:;">
<hgroup>
<h3>rChart \(MinBTL\)</h3>
</hgroup>
<article data-timings="">
<pre><code class="r">df &lt;- data.frame(
list(x=c(1,2:1000), minBTL=c(0,minBTL(2:1000))))
n1 &lt;- nPlot( minBTL ~ x, data = df, type = &quot;lineChart&quot;, height = 270, width = 800)
n1$yAxis( tickFormat = &quot;#!d3.format(&#39;,.2f&#39;)!#&quot;)
n1$chart( useInteractiveGuideline = TRUE )
n1
</code></pre>
<iframe src=assets/fig/unnamed-chunk-9.html seamless></iframe>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="" id="slide-10" style="background:;">
<hgroup>
<h3>Thanks</h3>
</hgroup>
<article data-timings="">
<ul>
<li>Ramnath Vaidyanathan</li>
<li>Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim</li>
</ul>
</article>
<!-- Presenter Notes -->
</slide>
<slide class="backdrop"></slide>
</slides>
<div class="pagination pagination-small" id='io2012-ptoc' style="display:none;">
<ul>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=1 title=''>
1
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=2 title='Constants - R Equivalents'>
2
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=3 title='Expected Maximum Sharpe \(E[max_N]\)'>
3
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=4 title='Plot \(E[max_N]\)'>
4
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=5 title='rChart \(E[max_N]\)'>
5
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=6 title='Check My Math with <em>Eq. 6</em>'>
6
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=7 title='Minimum Backtest Length \(MinBTL\)'>
7
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=8 title='Plot \(MinBTL\)'>
8
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=9 title='rChart \(MinBTL\)'>
9
</a>
</li>
<li>
<a href="#" target="_self" rel='tooltip'
data-slide=10 title='Thanks'>
10
</a>
</li>
</ul>
</div> <!--[if IE]>
<script
src="http://ajax.googleapis.com/ajax/libs/chrome-frame/1/CFInstall.min.js">
</script>
<script>CFInstall.check({mode: 'overlay'});</script>
<![endif]-->
</body>
<!-- Load Javascripts for Widgets -->
<script src="http://slidifylibraries2.googlecode.com/git/inst/libraries/widgets/bootstrap/js/bootstrap.min.js"></script>
<script src="http://slidifylibraries2.googlecode.com/git/inst/libraries/widgets/bootstrap/js/bootbox.min.js"></script>
<!-- MathJax: Fall back to local if CDN offline but local image fonts are not supported (saves >100MB) -->
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [['$','$'], ['\\(','\\)']],
processEscapes: true
}
});
</script>
<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/2.0-latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<!-- <script src="https://c328740.ssl.cf1.rackcdn.com/mathjax/2.0-latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script> -->
<script>window.MathJax || document.write('<script type="text/x-mathjax-config">MathJax.Hub.Config({"HTML-CSS":{imageFont:null}});<\/script><script src="http://slidifylibraries2.googlecode.com/git/inst/libraries/widgets/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML"><\/script>')
</script>
<script>
$(function (){
$("#example").popover();
$("[rel='tooltip']").tooltip();
});
</script>
<!-- LOAD HIGHLIGHTER JS FILES -->
<script src="http://slidifylibraries2.googlecode.com/git/inst/libraries/highlighters/highlight.js/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<!-- DONE LOADING HIGHLIGHTER JS FILES -->
</html>
title author license widgets github mode hitheme assets
Backtest Overfitting | Translated in R
TimelyPortfolio
by-nc-sa
mathjax
bootstrap
user repo
timelyportfolio
research_lopezdePrado
standalone
solarized_light

Constants - R Equivalents

# these are the constants referenced
# Euler–Mascheroni's constant
-digamma(1)
## [1] 0.5772
# euler constant
exp(1)
## [1] 2.718

Expected Maximum Sharpe $E[max_N]$

$$ \begin{aligned} (1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right] \end{aligned} $$

R Translation

### Eq. 4
emax <- function(N) {
  ( ( 1 + digamma(1) ) * qnorm( 1 - 1/N ) ) +  
      (-digamma(1) * qnorm( 1 - (1/N) * exp(-1)))
}

emax( N = 10 )  # should be about = 1.57 to match paper
## [1] 1.575

Plot $E[max_N]$

#implement upper bound
upbound <- function(N) {sqrt(2*log(N))}
#make a ugly plot for reasonableness check
curve(upbound, from = 1, to = 1000, col = "red", lty = 2)
curve(emax, from = 1, to = 1000, add=TRUE)
grid()

plot of chunk unnamed-chunk-4


rChart $E[max_N]$

require(rCharts)
df <- data.frame(
  list(x=c(1,2:1000), y=c(0,emax(2:1000))))
d1 <- dPlot( y ~ x, groups = "x", data = df, type = "line", height = 270, width = 800)
d1$xAxis(type = "addMeasureAxis",orderBy = "x",outputFormat = ",0.0f")
d1$yAxis( outputFormat = ".2f")
d1
<iframe src=assets/fig/unnamed-chunk-5.html seamless></iframe>

Check My Math with Eq. 6

### Try next example for Eq. 6
# if y = 5
# so solve for annualized Sharpe of 1
# says no more than 45 N should be tried

# first just do this to make sure I understand
N = 45
y = 5
emax( N ) * y^-0.5  #seems like on the right path
## [1] 0.9998

Minimum Backtest Length $MinBTL$

$$ \begin{aligned} \left(\frac{(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right]}{\overline{E[max_N]}}\right)^2 \end{aligned} $$

R Translation

#use emax from earlier for numerator
minBTL <- function( N, eMaxSharpe = 1 ) {
  (emax(N) / eMaxSharpe) ^ 2
}
#then this should equal 5 if correct
minBTL( N = 45, eMaxSharpe = 1 )
## [1] 4.998

Plot $MinBTL$

#make ugly plot for a reasonableness check
curve( minBTL, from = 1, to = 1000)

plot of chunk unnamed-chunk-8


rChart $MinBTL$

df <- data.frame(
  list(x=c(1,2:1000), minBTL=c(0,minBTL(2:1000))))
n1 <- nPlot( minBTL ~ x, data = df, type = "lineChart", height = 270, width = 800)
n1$yAxis( tickFormat = "#!d3.format(',.2f')!#")
n1$chart( useInteractiveGuideline = TRUE )
n1
<iframe src=assets/fig/unnamed-chunk-9.html seamless></iframe>

Thanks

  • Ramnath Vaidyanathan
  • Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim
---
title: Backtest Overfitting | Translated in R
author: TimelyPortfolio
license: by-nc-sa
widgets: [mathjax, bootstrap]
github: {user: timelyportfolio, repo: research_lopezdePrado}
mode: standalone
hitheme: solarized_light
assets:
css:
- "http://fonts.googleapis.com/css?family=Raleway:300"
- "http://fonts.googleapis.com/css?family=Oxygen"
--- dt:10
<style>
iframe{
height:450px;
width:900px;
margin:auto auto;
}
body{
font-family: 'Oxygen', sans-serif;
}
h1,h2,h3,h4 {
font-family: 'Raleway', sans-serif;
}
h3 {
background-color: #D4DAEC;
text-indent: 100px;
}
h4 {
text-indent: 100px;
}
iframe {height: 300px; width: 900px}
</style>
```{r message = F, warning = F, error = F, echo = F, tidy = F, cache = F}
require(knitr)
opts_chunk$set(
message = F,
warning = F,
error = F,
tidy = F,
cache = F,
fig.width = 11,
fig.height = 5
)
```
### Original Paper
<br>
<address>
<strong style="lineheight:40px;">Pseudo-Mathematics and Financial Charlatanism:</strong><p style="lineheight:40px;">The Effects of Backtest Overfitting on Out-of-Sample Performance</p>
<p class="muted" style="line-height:26px;">Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim<br>
October 7, 2013<br>
Available at SSRN: <a href="http://ssrn.com/abstract=2308659">http://ssrn.com/abstract=2308659</a>
</p>
</address>
---
### Constants - R Equivalents
```{r}
# these are the constants referenced
# Euler–Mascheroni's constant
-digamma(1)
# euler constant
exp(1)
```
---
### Expected Maximum Sharpe $E[max_N]$
$$
\begin{aligned}
(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right] \end{aligned}
$$
<h4>R Translation</h4>
```{r}
### Eq. 4
emax <- function(N) {
( ( 1 + digamma(1) ) * qnorm( 1 - 1/N ) ) +
(-digamma(1) * qnorm( 1 - (1/N) * exp(-1)))
}
emax( N = 10 ) # should be about = 1.57 to match paper
```
---
### Plot $E[max_N]$
```{r}
#implement upper bound
upbound <- function(N) {sqrt(2*log(N))}
#make a ugly plot for reasonableness check
curve(upbound, from = 1, to = 1000, col = "red", lty = 2)
curve(emax, from = 1, to = 1000, add=TRUE)
grid()
```
---
### rChart $E[max_N]$
```{r results = "asis"}
require(rCharts)
df <- data.frame(
list(x=c(1,2:1000), y=c(0,emax(2:1000))))
d1 <- dPlot( y ~ x, groups = "x", data = df, type = "line", height = 270, width = 800)
d1$xAxis(type = "addMeasureAxis",orderBy = "x",outputFormat = ",0.0f")
d1$yAxis( outputFormat = ".2f")
d1
```
---
### Check My Math with *Eq. 6*
```{r}
### Try next example for Eq. 6
# if y = 5
# so solve for annualized Sharpe of 1
# says no more than 45 N should be tried
# first just do this to make sure I understand
N = 45
y = 5
emax( N ) * y^-0.5 #seems like on the right path
```
---
### Minimum Backtest Length $MinBTL$
$$
\begin{aligned}
\left(\frac{(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right]}{\overline{E[max_N]}}\right)^2 \end{aligned}
$$
<h4> R Translation </h4>
```{r}
#use emax from earlier for numerator
minBTL <- function( N, eMaxSharpe = 1 ) {
(emax(N) / eMaxSharpe) ^ 2
}
#then this should equal 5 if correct
minBTL( N = 45, eMaxSharpe = 1 )
```
---
### Plot $MinBTL$
```{r}
#make ugly plot for a reasonableness check
curve( minBTL, from = 1, to = 1000)
```
---
### rChart $MinBTL$
```{r results = 'asis'}
df <- data.frame(
list(x=c(1,2:1000), minBTL=c(0,minBTL(2:1000))))
n1 <- nPlot( minBTL ~ x, data = df, type = "lineChart", height = 270, width = 800)
n1$yAxis( tickFormat = "#!d3.format(',.2f')!#")
n1$chart( useInteractiveGuideline = TRUE )
n1
```
---
### Thanks
- Ramnath Vaidyanathan
- Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment