Skip to content

Instantly share code, notes, and snippets.


zachlim98/ Secret

Created Nov 18, 2020
What would you like to do?
title categories
How to get the IBD50 list for FREE


The Investors' Business Daily is a newspaper covering the stock market, set up by William O'Neil in 1984. One of the most famous things that come from IBD is the IBD 50 - a list of the top 50 stocks that have been screened by the CAN SLIM method. The CAN SLIM method is a stock fundamental and technical screening methodology that was developed by William O'Niel.

So, one can do the screening manually using the methods described OR one can subscribe to the IBD. As part of one's IBD subscription, one would receive the weekly updates of the IBD 50. But, what if one just wants to know the IBD 50 without having the rest of the perks that come with an IBD subscription? Well, in 2015, innovator ETFs released an ETF (ticker: FFTY) that tracked the IBD50! And since the holdings of the ETF are available publicly, we can just scrap the holdings of FFTY in order to know each week what holdings are recommended by the CAN SLIM method

Disclaimer: An IBD subscription, I'm sure, comes with a lot more than the IBD50. This is merely a subset of what the subscription offers and so please feel free to subscribe to the full IBD if you find it helpful! For me, I felt I just needed the IBD50 list

Scrapping Process

Problems with BeautifulSoup


So, we all know that BeautifulSoup is the go-to when scrapping the web. However, I needed to scrap from and the issue with that is that stores the "holdings" in a JavaScript table. The second issue is that the initial page only displays 10 entries (out of the 50 stocks that are recommended under the IBD50 list), You can't really manipulate the webpage address to show the rest because, again, the holdings are stored in some JavaScript table. It was time to turn to the other favorite tool - selenium

Loading the webpage with Selenium

Because I was lazy and did not want to add the geckodriver to PATH (which you can find how to do here), I needed to specify my executable_path. (ps: the R in front is for Python to read it as a raw string and hence ignore the slash)

Once that was done, I simply set the URL for the FFTY holdings and used the driver to open the webpage.

Accessing a Dropdown box

Once that was done, I needed to access the dropdown list and change it such that it would show all 50 entries. The annoying about this, was that I kept getting an error that the element (the dropdown box) was blocked and not selectable! It turns out that the "Accept Cookies" form covered the dropdown box when the page initially loaded.


This was easily solved by using window.scroll to scroll it down about 400p (out of 1080p) so that the cookies form was out of the way. With that done, I simply had to find the dropdown box element and select '50'

Storing the data

This was the easiest part. After displaying all 50 symbols, I just had to use their CSS selector and pull the ticker symbol as well as their weights in the portfolio. I then used Yahoo Finance to pull their prices (just to make my life slightly easier).



So there you have it! Super easy, super fast way to get the IBD50 list without having to pay for a full subscription if you ever so desire! Thanks for reading!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment