Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
A brief description on how to use Morningstar's API

Morningstar API

If you're interested in grabbing Morningstar data but cash strapped there are ways to do it without actually paying for it.

Historical Price API

The following URL will get you access to all available historical Morningstar price data for the Ford Corporation

http://globalquote.morningstar.com/globalcomponent/RealtimeHistoricalStockData.ashx?ticker=F&showVol=true&dtype=his&f=d&curry=USD&range=1900-1-1|2014-10-10&isD=true&isS=true&hasF=true&ProdCode=DIRECT 

Now you will have every single daily piece of data for Ford between the dates of 01/01/1900 and 10/10/2014

Real Time Quote API

For more fine grained data for current market activity you can call the real time quote API.

http://quotespeed.morningstar.com/quote.jsp?&jsoncallback=jQuery172017541670752689242_1441265385395&preAfter=1&ty=D&mtype=ST&exch=126&ticker=F&stype=1&days=5&_tid=1441265386097&ver=1.6.0&f=1&instid=MSRT&sdkver=2.1.20150320&qs_wsid=2D2114F870F6B844CDF8271C99BB8C7A&_=1441265386099

Which will give you all data for the Ford Corporation within the last 5 days. Included is data for intraday trading.

Key Ratios

Get statistics for things like the following: Revenue, Net Income, Shares, Return on Equity, etc. all in CSV. For example; if I wanted to get all of the key ratios for Facebook I could do something like

http://financials.morningstar.com/ajax/exportKR2CSV.html?t=FB

You can then get your favorite CSV library to read the data for you.

Financials

Financial data for any ticker symbol you desire. Let's try Twitter (TWTR). First let's get the income statement

http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=TWTR&reportType=is&period=12&dataType=A&order=asc&columnYear=5&number=3

OK that's a lot of url parameters so let's break it down a bit:

  • reportType: is = Income Statement, cf = Cash Flow, bs = Balance Sheet
  • period: 12 for annual reporting, 3 for quarterly reporting
  • dataType: this doesn't seem to change and is always A
  • order: asc or desc (ascending or descending)
  • columnYear: 5 or 10 are the only two values supported
  • number: The units of the response data. 1 = None 2 = Thousands 3 = Millions 4 = Billions

So if we wanted to get the annual balance sheet information for TWTR we could request

http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=TWTR&reportType=bs&period=12&dataType=A&order=asc&columnYear=5&number=3

Likewise for Cash Flow;

http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=TWTR&reportType=cf&period=12&dataType=A&order=asc&columnYear=5&number=3

Disclaimer

This is in no way an official API. As such it is not supported by Morningstar the organization. Furthermore, I would advise to rate limit your downloads to be nice to Morningstar's servers. If not to be nice to Morningstar then to be kind to the other users of the unofficial API so as not to ruin a good thing.

Future

More fine grained analysis on each parameter forthcoming besides some wonky URL. Also, analysis on the response data.

@jp55

This comment has been minimized.

Copy link

commented Feb 11, 2016

How can I download data to an existing csv file?

@hahnicity

This comment has been minimized.

Copy link
Owner Author

commented Feb 18, 2016

Some additional programming perhaps. Check out stackoverflow if you are having problems

@GitHubDeepu

This comment has been minimized.

Copy link

commented Mar 30, 2016

Seems to giving only 5 years of data even if Col year is 10 when using VBA script. Works fine on browser. any suggestions?

@chrisapril

This comment has been minimized.

Copy link

commented Apr 11, 2016

Is the Real Time Quote API broken? I can not get any results with this url.

@hahnicity

This comment has been minimized.

Copy link
Owner Author

commented May 1, 2016

Possibly, haven't touched this in 9 months. If you want and you have access to Morningstar you can use the Chrome network debugging tool to help you figure out the API calls being used

@ppyaato

This comment has been minimized.

Copy link

commented May 12, 2016

How do I call a foreign company, lets say BP on the London stock exchange? Also is there a way to download the ownership information tab???

@mkomet

This comment has been minimized.

Copy link

commented May 21, 2016

I want to receive the last 3 movements (quarters) of a company - the exact date. Sometimes the Historical API works, and for certain tickers it doesn't (GOOGL, TWTR). Any idea why? Or how to fix it? Also, any way to know if it a company reports AM or PM?

@jvdheyden

This comment has been minimized.

Copy link

commented May 25, 2016

@ppyaato: you can get financial information for foreign companies by using their OTC market ticker. Look it up here for example: http://finance.yahoo.com/lookup;_ylc=X3oDMTIzNnFubjZjBGtleXcDRlJBTktMSU5HT0xEBG1pZANtZWRpYXF1b3Rlc3NlYXJjaARzZWMDZ2V0cXVvdGVzYnRuBHNsawNsb29rdXA-?s=FRANKLINGOLD
this won't help for getting the stock price from a certain stock market though

@dodgycode

This comment has been minimized.

Copy link

commented Nov 17, 2016

Good spot hahnicity.

Looks like they turned off the fund API. I tried this:
http://globalquote.morningstar.com/globalcomponent/FundQuote.aspx?ticker=fmagx&ver=h
(found at: http://globalquote.morningstar.com/globalcomponent/)

Does anyone know where of another URL for funds?

@bottleling

This comment has been minimized.

Copy link

commented Dec 5, 2016

Thanks for the urls :) Wrote some R code to read the data.
https://gist.github.com/anonymous/c7d9c19cc67e03641966064d1518ed41

@greedo18

This comment has been minimized.

Copy link

commented Dec 31, 2016

This is a great post! I'm having issues with foreign markets though. ie: Yahoo Finance finds information for Aphria Inc, ticker APH.V, but when I feed it into bottleling's R code it comes back NULL. Is there a different way to tell Morningstar that it's from the TSX venture exchange? I can find Aphria if I just search the ticker symbol in morningstar's website so they must have data....

@noobling

This comment has been minimized.

Copy link

commented Feb 4, 2017

Does anyone know if anyone has written about the parameters for the Real Time Quote API?

@robomotic

This comment has been minimized.

Copy link

commented Feb 16, 2017

What about the recommendation output, I found it to be like so (yes including the typo!)

http://quotes.morningstar.com/stockq/c-recommencation?&t=XNAS:MSFT&region=usa&culture=en-US&version=RET&cur=

however I am wondering if there is a hidden export feature like with the financials.

@shaon-chowdhury

This comment has been minimized.

Copy link

commented Feb 23, 2017

Does anyone know how to get adjusted close price and market cap using any API?

@mrhijodesupinshi

This comment has been minimized.

Copy link

commented Mar 14, 2017

hey guys. thanks @hahnicity. you are the man. I am only missing information from morningstar quote's page. seems like the provided link is no longer working. Anyone found a way around it?
im looking for: current price, market cap, sector and industry...

@cabezabolo

This comment has been minimized.

Copy link

commented Mar 28, 2017

Thanks for the info!, would be possible to make another JSON query to obtain the data from this page? http://financials.morningstar.com/valuation/price-ratio.html?t=KO (Historical P/E ratio would be very useful

Edit: after some debugging in "Network" tab in the Chrome Dev Tools I've got 4 new URLS:

Current Valuation: http://financials.morningstar.com/valuate/current-valuation-list.action?&t=FB&region=usa&culture=en-US&cur=&adsFlag=true&_=1490717022553

Forward Valuation: http://financials.morningstar.com/valuate/forward-valuation-list.action?&t=FB&region=usa&culture=en-US&cur=&adsFlag=true&_=1490717022554

Valuation History: http://financials.morningstar.com/valuate/valuation-history.action?&t=FB&region=usa&culture=en-US&cur=&type=price-earnings&_=1490717022555 --> That's what I needed!! Cheeeeeeers!

Dividend Yield (returns a Javascript, values are into "barValues"): http://financials.morningstar.com/valuate/valuation-yield.action?&t=FB&region=usa&culture=en-US&cur=&_=1490717022555

@agentmorris

This comment has been minimized.

Copy link

commented Apr 29, 2017

I'm interested in using this approach to retrieve expense ratios for mutual funds... does anyone know the right URL and parameters to retrieve the equivalent of:

http://financials.morningstar.com/fund/expense.html?t=FINPX

...without HTML-scraping?

Thanks!

@ucb

This comment has been minimized.

Copy link

commented May 3, 2017

I am struggling with the Historical Price API.

In particular, changing currency has no effect on the output. Everything comes out the same, whatever currency code is specified. I have played around with some other currency options from different other forms, but that had no effect either. Has anyone been able to decipher all fields of the url? Any pointers would be greatly appreciated!

@cabezabolo: Chrome Dev Tools: how do you use it for this purpose? Do you think it could be useful for what I am trying to figure out?

NB: If you will be using the code by anonymous, you will need to change "sub" to "gsub", set 1900, jan-1, as the origin, and if your data does not contain all fields, you will need to fix column name assignment.

@cabezabolo

This comment has been minimized.

Copy link

commented May 5, 2017

@ucb I think that morningstar has only data for the tickers's official currency (if the company has shares in different markets, also will have different tickers for each one, example: FB (NYSE / USD) / FB2A (Xetra / EUR), but you can convert it easily to another currency with =GOOGLEFINANCE("CURRENCY:GBPUSD") in Google Docs, for example.

By using Chrome Dev tools, after reloading the webpage, you'll can see all the HTTP requests in the "Headers" tab, clicking on the items in the left list. You can filter in the upper bar, clicking on JS to get only JSON requests, for example, and I try to look any request to the morningstar subdomains (i.e. financials.morningstar.com, quotes.morningstar.com, investors.morningstar.com and I guess that will be more, but I haven't found them yet). Here we have a request example on the "Quote" tab in morningstar.com

morningstar_quote

http://mschart.morningstar.com/chartweb/defaultChart?type=getcc&secids=0P000001BW;ST&dataid=8226&startdate=1900-01-01&enddate=2017-05-05&currency=&format=1&callback=jQuery164015960861115569203_1494006365716&_=1494006367815

In this case, it returns price data from 06/01/1972 !!! (Coca-Cola), but I would prefer to use GOOGLEFINANCE formulas for the price, you can query the data for years...

But the requests will not always be JSON queries, sometimes we'll need to navigate through the "All" tab between a lot of garbage, some patience required.

I'm trying to load Financials / Balance Sheet / Cash Flow for any ticker in a Google Docs spreadsheet file, but this will require some hours to make it work, for sure :)

@agentmorris I've tried but I didn't found the correct request for that information.

@ar-anvd

This comment has been minimized.

Copy link

commented Jun 29, 2017

A question, there is any way of getting a fund name for the id ? For example get the name Invesco Global Smaller Coms Eq E EUR Acc for this F00000NGSG

@gottlieben

This comment has been minimized.

Copy link

commented Aug 1, 2017

&dataType=A means as reported, &dataType=R as restated (relevant for financial statements)

@raydizzle29

This comment has been minimized.

Copy link

commented Aug 25, 2017

Very small update to bottleling's awesome R script to pull in all prices through today, rather than a fixed date in the past:

today <- as.character(Sys.Date())

myticker<-"FB"
url.histprice<-function(x){ return(paste0("http://globalquote.morningstar.com/globalcomponent/RealtimeHistoricalStockData.ashx?ticker=",x,"&showVol=true&dtype=his&f=d&curry=USD&range=1900-1-1|",today,"&isD=true&isS=true&hasF=true&ProdCode=DIRECT"))}

@DeepthiKR444

This comment has been minimized.

Copy link

commented Oct 9, 2017

Hi,
Is there is any web services call or API to retrieve mutual fund list available?

@puppy79

This comment has been minimized.

Copy link

commented Oct 13, 2017

Great post here. I am trying to get the key ratio. kind of new to Python. how can i use the following and use python to automatically pull the key ratio of a list of tickers and save to csv file? Thanks for any help.

http://financials.morningstar.com/ajax/exportKR2CSV.html?t=x

@rsvp

This comment has been minimized.

Copy link

commented Oct 16, 2017

So how does Morningstar API compare to other data vendors?

Please respond at rsvp/fecon235#7
since gists do not have decent notifications.
We are looking for integration with Python pandas DataFrame.

@fedsanchez

This comment has been minimized.

Copy link

commented Nov 9, 2017

Do you know if there is a way to get a list of current (not real time) stock prices for a series of canadian stocks?

@acemanhattan

This comment has been minimized.

Copy link

commented Nov 28, 2017

Is anyone else able to get 10 year history? I'm only able to get 5.

@jappeace

This comment has been minimized.

Copy link

commented Dec 24, 2017

To get foreign exchanges the yahoo thing didn't work for me, what did work was going to: http://financials.morningstar.com and then enter the name without searching so you get the pop up menu(eg BHP), and selecting the right one (I was interested in the australian one. The form then generates the url for you, In my case:
http://financials.morningstar.com/income-statement/is.html?t=XASX:BHP&region=AUS&ops=clear

@thekidcalvin

This comment has been minimized.

Copy link

commented Dec 29, 2017

I use this API to parse mutual fund price and dividend history into an Access database, but unfortunately it doesn't pull in capital gains distributions information; only distributions classified as "dividends". Does anyone know of another free API that might provide both cap gains and dividend history? Or in lieu of that, one that will provide adjusted price so that I can calculate total return directly?

Thanks for this post btw! This API has been a lifesaver.

@mmontefl

This comment has been minimized.

Copy link

commented Feb 6, 2018

Found your code via a google search. Searched the https://developer.morningstar.com/# website and was wondering if any of this requires a paid subscription. I am trying to develop an app, and would like to keep costs to minimum. I see Quandl and alpha Vantage and IEX seem to offer free API data. I am still in discovery phase. Have suggestions?
My initial plan is to use Python and store some of the info in mongo.

@lawonga

This comment has been minimized.

Copy link

commented Feb 17, 2018

Any way for dividend data to be in json format?

@nikhilnagpal123

This comment has been minimized.

Copy link

commented Mar 26, 2018

hi, is it legal to use this API via PHP? in the robots.txt file they disallowed all the robots

@MichelangeloConserva

This comment has been minimized.

Copy link

commented Apr 15, 2018

For those who are interested in Python code which scrape all the data present on the main page of Morningstar (ex. 'http://www.morningstar.com/stocks/xnas/aapl/quote.html') I will release a package which will be able to do that and lot of more interesting stuff in a month or less.

@yewmun2008

This comment has been minimized.

Copy link

commented Apr 20, 2018

Anyone have documentation for Historical Price API?
http://globalquote.morningstar.com/globalcomponent/RealtimeHistoricalStockData.ashx?
ticker=F&showVol=true&dtype=his&f=d&curry=USD&range=1900-1-1|2014-10-10&isD=true&isS=true&hasF=true&ProdCode=DIRECT

I wonder what those parameters mean, and the return results for date became number instead of date format, anyone have a clue how to make it become date format?

@chmorik

This comment has been minimized.

Copy link

commented May 11, 2018

I found a way to get Fair Value Estimate and other goodies
https://api-global.morningstar.com/sal-service/v1/stock/morningstarTake/v3/0P000002DO/analysisData
For Example: GE
{
"ticker": "GE",
"isQuan": false,
"userType": "Premium",
"valuation": {
"fairValue": "19",
"fairValueDate": "2018-04-23T00:00:00.000",
"assessment": "Undervalued",
"assessmentDate": "2018-05-10T00:00:00.000",
"uncertainty": "High",
"moat": "Wide",
"moatDate": "2018-05-10T00:00:00.000",
"moatTrendEvaluate": "Stable",
"premiumDisc": "Discount",
"stewardship": "Poor",
"stewardshipDate": "2018-05-10T00:00:00.000",
"premDiscDelta": "-23",
"oneStar": "29.45",
"oneStarDate": "2018-05-10T00:00:00.000",
"fiveStar": "11.4",
"fiveStarDate": "2018-05-10T00:00:00.000",
"fairValCurrency": "USD",
"bf2": "29.45",
"bf3": "21.85000",
"bf4": "16.15000",
"bf5": "11.4",
"lastClose": "14.69",
"startRating": "4",
"lastCloseCurrency": "USD"
},
"companyProfile": "General Electric is a diversified manufacturer that operates seven separate businesses in its industrial segment: power, oil and gas, renewable energy, lighting, aviation, healthcare, and transportation. The company also provides specialty industrial financing through its GE Capital division."
}

You'll need 3 headers
Apikey
X-SAL-ContentType
X-API-REALTIME-E
That you can take from you're browser

I'm premium and I'm currently working on a api to query and get the premium data from morning star

@chmorik

This comment has been minimized.

Copy link

commented May 12, 2018

Hi,

I've build an API that retrieve morning star perimum analysis data

You can query any ticker you wish

for example

https://morningstar-api.herokuapp.com/analysisData?ticker=GE

will respond with

{
"ticker": "GE",
"isQuan": false,
"userType": "Premium",
"valuation": {
"fairValue": "19",
"fairValueDate": "2018-04-23T00:00:00.000",
"assessment": "Undervalued",
"assessmentDate": "2018-05-10T00:00:00.000",
"uncertainty": "High",
"moat": "Wide",
"moatDate": "2018-05-10T00:00:00.000",
"moatTrendEvaluate": "Stable",
"premiumDisc": "Discount",
"stewardship": "Poor",
"stewardshipDate": "2018-05-10T00:00:00.000",
"premDiscDelta": "-23",
"oneStar": "29.45",
"oneStarDate": "2018-05-10T00:00:00.000",
"fiveStar": "11.4",
"fiveStarDate": "2018-05-10T00:00:00.000",
"fairValCurrency": "USD",
"bf2": "29.45",
"bf3": "21.85000",
"bf4": "16.15000",
"bf5": "11.4",
"lastClose": "14.69",
"startRating": "4",
"lastCloseCurrency": "USD"
},
"companyProfile": "General Electric is a diversified manufacturer that operates seven separate businesses in its industrial segment: power, oil and gas, renewable energy, lighting, aviation, healthcare, and transportation. The company also provides specialty industrial financing through its GE Capital division."
}

Enjoy :)

@dmatranga

This comment has been minimized.

Copy link

commented Jun 3, 2018

@chmorik This is useful, thanks

@dmatranga

This comment has been minimized.

Copy link

commented Jun 3, 2018

Thanks to @hahnicity for original post also, good stuff!

@MiniXC

This comment has been minimized.

Copy link

commented Jul 29, 2018

Thank you for this write-up. About the key-ratios: can the dates, e.g. 2017-09 be taken at face value? Or are those the results of that company's quarter that get released later than the given date?

@chenp7

This comment has been minimized.

Copy link

commented Jul 31, 2018

I was using the API for a couple of month and works absolutely fine. But since this morning, I am getting 404 error (e.g. http://globalquote.morningstar.com/globalcomponent/RealtimeHistoricalStockData.ashx?ticker=F&showVol=true&dtype=his&f=d&curry=USD&range=1900-1-1|2014-10-10&isD=true&isS=true&hasF=true&ProdCode=DIRECT
). Anyone having the same issue?

I also noticed there are some serious data data discrepancy (over 400 symbols has apparent issue, for example: low price > open price). I have reported this to Morningstar and they are investigating. Anyone noticed the same thing?

@skilbjo

This comment has been minimized.

Copy link

commented Jul 31, 2018

@chenp7 thanks for your report, I am also experiencing that the api is down

@aokellermann

This comment has been minimized.

Copy link

commented Jul 31, 2018

It also 404's for me. Does anyone know a good mutual fund api in the meantime before IEX releases v2.0?

@brianzzzasd

This comment has been minimized.

Copy link

commented Aug 29, 2018

Is there any update for real time stock quote?

@abalter

This comment has been minimized.

Copy link

commented Sep 28, 2018

Same problem here. I'm searching around

@jose963

This comment has been minimized.

Copy link

commented Nov 9, 2018

How do I get Morningstar ticker for an ISIN reference for a stock (not funds) ?
Or, is it possíble to search stocks data using ISIN instead of ticker?

@skywalker219

This comment has been minimized.

Copy link

commented Nov 28, 2018

Would appreciate any information about downloading JSON historical price data from Morningstar (funds, etfs, stocks, etc).

Getting 502 ERROR when using the following request:
http://globalquote.morningstar.com/globalcomponent/RealtimeHistoricalStockData.ashx?ticker=F&showVol=true&dtype=his&f=d&curry=USD&isD=true&isS=true&hasF=true&ProdCode=DIRECT

@jackcballinger

This comment has been minimized.

Copy link

commented Jan 5, 2019

@ucb I think that morningstar has only data for the tickers's official currency (if the company has shares in different markets, also will have different tickers for each one, example: FB (NYSE / USD) / FB2A (Xetra / EUR), but you can convert it easily to another currency with =GOOGLEFINANCE("CURRENCY:GBPUSD") in Google Docs, for example.

By using Chrome Dev tools, after reloading the webpage, you'll can see all the HTTP requests in the "Headers" tab, clicking on the items in the left list. You can filter in the upper bar, clicking on JS to get only JSON requests, for example, and I try to look any request to the morningstar subdomains (i.e. financials.morningstar.com, quotes.morningstar.com, investors.morningstar.com and I guess that will be more, but I haven't found them yet). Here we have a request example on the "Quote" tab in morningstar.com

morningstar_quote

http://mschart.morningstar.com/chartweb/defaultChart?type=getcc&secids=0P000001BW;ST&dataid=8226&startdate=1900-01-01&enddate=2017-05-05&currency=&format=1&callback=jQuery164015960861115569203_1494006365716&_=1494006367815

In this case, it returns price data from 06/01/1972 !!! (Coca-Cola), but I would prefer to use GOOGLEFINANCE formulas for the price, you can query the data for years...

But the requests will not always be JSON queries, sometimes we'll need to navigate through the "All" tab between a lot of garbage, some patience required.

I'm trying to load Financials / Balance Sheet / Cash Flow for any ticker in a Google Docs spreadsheet file, but this will require some hours to make it work, for sure :)

@agentmorris I've tried but I didn't found the correct request for that information.

for me this returns the date and volume, rather than any pricing information - any idea how to get pricing information, and is there a mapping somewhere for funds/shares and their respective IDs?

@G4rpez

This comment has been minimized.

Copy link

commented Jan 13, 2019

@jackcballinger I think there's no specific dataIDs for each fund/etf, dataIDs are grouped depending on region/type. For European funds Asset Managers I've found 8216 id working for all.

I've also found out that Morningstar's Ticker must be UPPERCASE in link, otherwise you'll get empty JSON objects.

Moreover strings after last date (e.g. "&currency=&format=1&callback=jQuery164015960861115569203_1494006365716&_=1494006367815") can be removed

@iceliu

This comment has been minimized.

Copy link

commented Feb 22, 2019

@chmorik Thanks for your post. Is it passable get fairValue from https://developer.morningstar.com/apis , and can you share code that you put in herokuapp.com? I am also a Premium user.

@otis632

This comment has been minimized.

@MikeCush

This comment has been minimized.

Copy link

commented Mar 27, 2019

Anyone find a fix to get 10 year data? I seem to only be able to get 5 like a few other people on this thread. Any recommendations?

@psthlm

This comment has been minimized.

Copy link

commented Mar 28, 2019

I have used the API:
http://financials.morningstar.com/ajax/exportKR2CSV.html?t=V

for a year or so. But today it stopped working. Temporary or does someone have any idea why?

@Free008

This comment has been minimized.

Copy link

commented Mar 28, 2019

I think everything in http://financials.morningstar.com/ajax/ is broken, including financials.

@MortenSDK

This comment has been minimized.

Copy link

commented Mar 28, 2019

I created an account today just to comment, that this morning it seemed to be broken.
Apparently I am not the only one having issues.
Maintenance, or is it more likely that it shut down for good?

I'd hate for it to be gone!

@Free008

This comment has been minimized.

Copy link

commented Mar 28, 2019

Yeah me too. Morningstar is being really silly - I think they rejigged the web pages as well, and the back end got changed. They just want you to come to their site and buy their funds, not make your own investment decisions by downloading the data and crunch the numbers, so they cut off their API. Even though 50 years academic research shows that no fund can consistently beat index, and therefore their fund ratings have no empirical support. That's the reason why they will give you 5 years financials, but theirs no easy way to get the 10 year even if you are a subscriber. I heard it from the horse's mouth - "its the business model". I mean some of their financials take weeks or months after the report date, so its nothing special. The guys crunching the APIs everyday and building a "Morningstar Fundamentals" data business probably didn't help either.

@ivopbernardo

This comment has been minimized.

Copy link

commented Mar 28, 2019

Hoping that it is something that it will be solved but knowing that there is a probability that this service will not be up for a long time, does anyone know any alternative to scrap margins and fundamentals about public companies for free?

@tonkolviktor

This comment has been minimized.

Copy link

commented Mar 28, 2019

Hello,

check out this one: https://stockrow.com/MMM/financials/income/annual

Cheers

@Free008

This comment has been minimized.

Copy link

commented Mar 28, 2019

Hello,

check out this one: https://stockrow.com/MMM/financials/income/annual

Cheers

That's an interesting site, but not consistent. I cant get quarterly consistently for example. I think I'm going to go to Sharadar for fundamentals and pay for it.

@marcliu11

This comment has been minimized.

Copy link

commented Mar 29, 2019

having the same issues, was pulling data from morningstar for the better part of 3 years using:

=IMPORTDATA("http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t="&Summary!A7&"&region=usa&culture=en-US&cur=&order=desc")

with google sheets, works like a charm. but now it seems dead. just about 24hrs ago..

needed this urgently to conduct a class tommorow on valuation. Are there are any options anyone can help with? I don't mind paying a subscription service so that my students can employ use of the spreadsheet.

@MortenSDK

This comment has been minimized.

Copy link

commented Mar 29, 2019

Hello,

check out this one: https://stockrow.com/MMM/financials/income/annual

Cheers

The stockrow data can be downloaded as .xls - You could script a download to google sheets, convert to google sheets, and then paste the values into your worksheet. It's a clunky workaround, but it might work.

Anyone knows where this data comes from? is it reliable?

@marcliu11

This comment has been minimized.

Copy link

commented Mar 29, 2019

Thanks MortenSDK, sounds like a great idea, but I have just basic knowledge of programming. Any tips?

@marcliu11

This comment has been minimized.

Copy link

commented Mar 29, 2019

what i did previously was to pull in the entire morningstar financials into google sheets. how do i do the same with stockrow?

@Free008

This comment has been minimized.

Copy link

commented Mar 29, 2019

what i did previously was to pull in the entire morningstar financials into google sheets. how do i do the same with stockrow?

I think you can try the API when you click on download excel, for example:

https://stockrow.com/api/companies/AAPL/financials.xlsx?dimension=MRQ&section=Income%20Statement&sort=desc

@marcliu11

This comment has been minimized.

Copy link

commented Mar 29, 2019

I find stockrow inconsistent, figures wise and also the number of years. For example WMT has 9 years of data, whilst some 10 years and etc. So I guess it'll not work out. Thanks anyway!

@ZeccaLehn

This comment has been minimized.

Copy link

commented Mar 29, 2019

Until Morningstar fixes this, I went ahead and created a link to my Python Selenium solution here: https://colab.research.google.com/drive/1Q-WdtVQ4A-UOZv0poJzRIO6AtcaqQY1D

@ZeccaLehn

This comment has been minimized.

Copy link

commented Mar 29, 2019

For more on advanced headless scraping in Python -- which walks through Selenium / Headless scraping-- check out my recent post https://towardsdatascience.com/virtual-headless-distributed-6c7ebef25a96 (Virtual, Headless, and Distributed (Oh My!)

@ZeccaLehn

This comment has been minimized.

Copy link

commented Mar 30, 2019

Until Morningstar fixes this, I went ahead and created a link to my Python Selenium solution here: https://colab.research.google.com/drive/1Q-WdtVQ4A-UOZv0poJzRIO6AtcaqQY1D

Sterling job. I've been working on something using Selenium but not quite there. Any way to amend your code to download the CSV directly into a python variable instead of the local PC?

Thanks ZeccaLehn!

It takes a min 2 sec to allow the headless browser to download. Don't think the pd.read_csv and delete of csv will take much longer. A quicker workaround would be to use the css selector to collect from the tabs in the ratio link.

@MortenSDK

This comment has been minimized.

Copy link

commented Mar 30, 2019

I managed to find this, which would be useful for my needs. I can't find a way to parse the html - any suggestions?
http://financials.morningstar.com/finan/financials/getFinancePart.html?t=IBM&region=usa&culture=en-US&ops=clear

@nachobec

This comment has been minimized.

Copy link

commented Mar 30, 2019

I write an email to morningstar but they answer saying what my client ID was, my morningstar account, etc. I explained it had nothing to see with that, I was just a free user. Now I'm waiting for their answer again.

@KNKalinin

This comment has been minimized.

Copy link

commented Mar 30, 2019

I managed to find this, which would be useful for my needs. I can't find a way to parse the html - any suggestions?
http://financials.morningstar.com/finan/financials/getFinancePart.html?t=IBM&region=usa&culture=en-US&ops=clear

import pandas as pd
tables = pd.read_html('http://financials.morningstar.com/finan/financials/getFinancePart.html?t=IBM&region=usa&culture=en-US&ops=clear')
tables[0]

@edmondlowjy

This comment has been minimized.

Copy link

commented Mar 31, 2019

I write an email to morningstar but they answer saying what my client ID was, my morningstar account, etc. I explained it had nothing to see with that, I was just a free user. Now I'm waiting for their answer again.

Seriously why in the world would you write to morningstar for this?! If they never meant to expose this they may just take down the entire site entirely, no thanks to you.

@edmondlowjy

This comment has been minimized.

Copy link

commented Mar 31, 2019

Guys can we please be cooperative here. Do refrain from explicitly posting how you're retrieving the data lest it gets taken away. Let's not give ourselves away and lose a valuable data resource.

@MortenSDK

This comment has been minimized.

Copy link

commented Apr 1, 2019

I managed to find this, which would be useful for my needs. I can't find a way to parse the html - any suggestions?
http://financials.morningstar.com/finan/financials/getFinancePart.html?t=IBM&region=usa&culture=en-US&ops=clear

import pandas as pd
tables = pd.read_html('http://financials.morningstar.com/finan/financials/getFinancePart.html?t=IBM&region=usa&culture=en-US&ops=clear')
tables[0]

thanks! I will resort to this, if i cant find a way to import it to google sheets.

@Toscan6

This comment has been minimized.

Copy link

commented Apr 1, 2019

They added an additional layer of security, the link is still working, i have performed a HTTP Trace on the download button and everything indicates that we need to be logged in the site to be able to get the data.

I have been struggling since then to create a google script that would log me in to morningstar.com

This is what i have been able to do so far:

function loginToSite(){
function Navigator(baseUrl){
var nav = new Navigator.Navigator("https://www.morningstar.com/members/login.html");
nav.setSaveCookies(true);
nav.setCookieUsername(email);
// login form at http://www.example.com/login
nav.setLoginPath("https://www.morningstar.com/members/login.html");
// will output all URLs, headers, cookies and payloads via Logger
nav.setDebug(true);
// for static login forms, this payload will be submitted during automatic log-in (if enabled)
nav.setLoginPayload({
"E-mail Adress":"your email", //change this.
"Password":"password" //and change this. done. no need to enter anything elsewhere.
});
// only logged-out pages contain this; if we see this, we know we're logged out
nav.setLogoutIndicator("password_reset");
// if you request /home, and Navigator happens to find itself logged out and logs in again,
// setting this to true will make Navigator re-fetch /home right after the re-login
nav.setRefetchOnLogin(true);
// try #1; will automatically log in if stored cookies are already expired
var str = nav.doGet("daily");
if (str.indexOf("Get daily prize now") > 0) {
str = nav.doGet("daily"); // try #2
if (str.indexOf("Get daily prize now") > 0) {
// notify failure
}
}
}}

@nachobec

This comment has been minimized.

Copy link

commented Apr 1, 2019

Hi Toscan6 ,
Like in the example of the original post, I write in a cell of Google Sheets the following:
=importdata("http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=TWTR&reportType=bs&period=12&dataType=A&order=asc&columnYear=5&number=3")

You say now it does not work because of a new security layer. However, I have sign in in Morningstar with my account (same gmail email as my google sheets google account). And it still does not work....

Where should we paste your code? If I create a Google Apps Script in my Google Drive, how should I connect that code with my google sheet where I have the =importdata(...) formula?

(sorry, my question may be very basic, but it would help me too much. Thanks in advance).

@dehaosun

This comment has been minimized.

Copy link

commented Apr 2, 2019

I have tried jxBroswer to simulate chrome in java, and it worked. Now It cost 3-5 secs to wait whole page load and then invoke javascript "exportKeyStat2CSV" and get a single csv file. So I am working on handling cookies on multi threads and organize summary data and then upload them to googleSheet . Since the summary data is on googleSheet, you can just easily use "vlookup" to locate whatever you need!

PS. the program does not login to MorningStar.

@dehaosun

This comment has been minimized.

Copy link

commented Apr 2, 2019

They added an additional layer of security, the link is still working, i have performed a HTTP Trace on the download button and everything indicates that we need to be logged in the site to be able to get the data.

I have been struggling since then to create a google script that would log me in to morningstar.com

This is what i have been able to do so far:

function loginToSite(){
function Navigator(baseUrl){
var nav = new Navigator.Navigator("https://www.morningstar.com/members/login.html");
nav.setSaveCookies(true);
nav.setCookieUsername(email);
// login form at http://www.example.com/login
nav.setLoginPath("https://www.morningstar.com/members/login.html");
// will output all URLs, headers, cookies and payloads via Logger
nav.setDebug(true);
// for static login forms, this payload will be submitted during automatic log-in (if enabled)
nav.setLoginPayload({
"E-mail Adress":"your email", //change this.
"Password":"password" //and change this. done. no need to enter anything elsewhere.
});
// only logged-out pages contain this; if we see this, we know we're logged out
nav.setLogoutIndicator("password_reset");
// if you request /home, and Navigator happens to find itself logged out and logs in again,
// setting this to true will make Navigator re-fetch /home right after the re-login
nav.setRefetchOnLogin(true);
// try #1; will automatically log in if stored cookies are already expired
var str = nav.doGet("daily");
if (str.indexOf("Get daily prize now") > 0) {
str = nav.doGet("daily"); // try #2
if (str.indexOf("Get daily prize now") > 0) {
// notify failure
}
}
}}

Hi Toscan, I have thought the solution in google apps.
however the free usage limitation ( 4.5 mins per/call and 1 hours per day ) knock me back to local java client.
Also I have done multi fetch csv structure in google App before, the performance sometimes is poor.
If you are using free usage with your google account, maybe you can start to think about the performance.

@Alzheimancer

This comment has been minimized.

Copy link

commented Apr 2, 2019

The guy reported this bugs to the Morningstar because you cannot download the data is truly idiot . This API is not free but you accidentally told them, so now no one can access this valuable data anymore.

@Toscan6

This comment has been minimized.

Copy link

commented Apr 2, 2019

Hey guys,

I see that some progress has been done. I am going to give a try to jxBroswer. I feel that i am already giving up on the dynamic function and will content myself with manual job. Instead of 2 seconds, it will take two minutes and i will survive.

I will continue to keep a close eye on the progress made here, until then, good luck!

@otis632

This comment has been minimized.

Copy link

commented Apr 2, 2019

The guy reported this bugs to the Morningstar because you cannot download the data is truly idiot . This API is not free but you accidentally told them, so now no one can access this valuable data anymore.

Seconded

@nickkraakman

This comment has been minimized.

Copy link

commented Apr 4, 2019

Hmm this is a major setback... Looking at the network tab of the key ratios page, I found the following link that returns the data of the page in JSONP format:

https://financials.morningstar.com/finan/financials/getKeyStatPart.html?&t=AAPL&region=usa&culture=en-US&cur=&order=asc

It is not as easy to parse as the csv file, but the data is there.

Anyone found an easier workaround yet?

EDIT:
I just wrote this PHP script to clean the data so that I can use it with the ImportHTML() function in Google Spreadsheets:

<?php
// Check if ticker is set in url
if(isset($_GET['ticker']) && $_GET['ticker'] != "" && isset($_GET['page']) && $_GET['page'] != "") {
	$ticker = $_GET['ticker'];
	$page = $_GET['page'];
} else {
	echo "Invalid syntax";
	exit();
}

$page_type = "getKeyStatPart";
if ($page == "stats") {
    $page_type = "getKeyStatPart";
} else if ($page == "financials") {
    $page_type = "getFinancePart";
}


/**
 * CURL alternative to PHPs file_get_contents()
 */
function curl_get_contents($url) {

    $ch = curl_init();
  
    curl_setopt( $ch, CURLOPT_AUTOREFERER, TRUE );
    curl_setopt( $ch, CURLOPT_HEADER, 0 );
    curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
    curl_setopt( $ch, CURLOPT_URL, $url );
    curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, TRUE );
  
    $data = curl_exec( $ch );
    curl_close( $ch );
  
    return $data;
}

$json = curl_get_contents('https://financials.morningstar.com/finan/financials/'.$page_type.'.html?&t='.$ticker.'&region=usa&culture=en-US&cur=&order=asc&');
$decoded_json = json_decode($json, TRUE);
$html = $decoded_json["componentData"];
$clean_html = str_replace('<tr class="hr"><td colspan="12"></td></tr>', '', $html);

echo $clean_html;
@Toscan6

This comment has been minimized.

Copy link

commented Apr 4, 2019

There is a way to do this playing with the Stylesheet and Python. It is mentioned above @ZeccaLehn . Maybe, something to dig on gscript, if gscript can manage CSS to create the table, that could be an interesting path to take.

@Toscan6

This comment has been minimized.

Copy link

commented Apr 4, 2019

@nickkraakman , i think i recognize this handler, you are the guy who made the buffett spreadsheet ;) . Looks like you can use the importhtml function in google sheets, it works but it keeps the markdown, a little script could remove this easily.

@Toscan6

This comment has been minimized.

Copy link

commented Apr 4, 2019

Looks like i am into something, i am looking at getting this into my google sheet

https://gist.github.com/paulgambill/cacd19da95a1421d3164

I will keep you guys posted.

@clarchoi

This comment has been minimized.

Copy link

commented Apr 7, 2019

@nickkraakman Very useful to get key ratio page! Many thanks! How to convert to JSONP format for income statement ?
http://financials.morningstar.com/income-statement/is.html?t=aapl&region=usa&culture=en-US

@antoinevulcain

This comment has been minimized.

Copy link

commented Apr 9, 2019

You can find all income statement, balance sheet statement and cash flow statetement in JSON format and Excel format here :

https://financialmodelingprep.com/api/financials/income-statement/AAPL

They have also quaterly financial statement data, you can find more information in their documentation here :

https://financialmodelingprep.com/developer/docs

@clarchoi

This comment has been minimized.

Copy link

commented Apr 10, 2019

@antoinevulcain Thanks! Great but it only covers US stocks. Morningstar.com provides more international markets.

@dehaosun

This comment has been minimized.

Copy link

commented Apr 11, 2019

Here are my sample code to download csv with jxBrowser, and it works.
public static void main(String[] args) {

		if (Environment.isMac()) {
        BrowserCore.initialize();
    }
	
	
    Browser browser = new Browser(BrowserType.HEAVYWEIGHT);
    BrowserView view = new BrowserView(browser);

// JFrame frame = new JFrame("JxBrowser");
// frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
// frame.add(view, BorderLayout.CENTER);
// frame.setExtendedState(JFrame.MAXIMIZED_BOTH);
// frame.setLocationRelativeTo(null);
// frame.setVisible(true);

    JFrame frame = new JFrame();
    frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
    frame.add(view, BorderLayout.CENTER);
    frame.setSize(700, 500);
    frame.setLocationRelativeTo(null);
    frame.setVisible(false);

    NetworkService networkService = browser.getContext().getNetworkService();
    networkService.setResourceHandler(new ResourceHandler() {
        @Override
        public boolean canLoadResource(ResourceParams params) {

// System.out.println("URL: " + params.getURL());
// System.out.println("Type: " + params.getResourceType());
boolean isNotImageType = false;
if(
params.getResourceType() != ResourceType.IMAGE
// && params.getResourceType() != ResourceType.STYLESHEET
// && params.getResourceType() != ResourceType.MAIN_FRAME
// && params.getResourceType() != ResourceType.XHR
) isNotImageType = true;

            if (isNotImageType) {
                return true;    // Cancel loading of all images
            }
            return false;
        }
    });
    
    

    
    
    tryDown(browser,"http://financials.morningstar.com/ratios/r.html?t=FB", "FB");
    tryDown(browser,"http://financials.morningstar.com/ratios/r.html?t=DIS", "DIS");
    
    
    //browser.loadURL("http://financials.morningstar.com/ratios/r.html?t=FB");
    
   
    

    
}

private static void tryDown(Browser browser, String url, String fileName) {
	 Browser.invokeAndWaitFinishLoadingMainFrame(browser, new Callback<Browser>() {
            @Override
            public void invoke(Browser value) {
                value.loadURL(url);
            }
        });
        
	 
	 
        browser.setDownloadHandler(new DownloadHandler() {
            public boolean allowDownload(DownloadItem download) {
            		download.setDestinationFile(new File("/Stock_Paser/Stock_PP/"+fileName+".csv"));
                download.addDownloadListener(new DownloadListener() {
                    public void onDownloadUpdated(DownloadEvent event) {
                        DownloadItem download = event.getDownloadItem();
                        if (download.isCompleted()) {
                            System.out.println("Download is completed!");
                        }
                    }
                });
                System.out.println("Destination file: " +
                        download.getDestinationFile().getAbsolutePath());
                return true;
            }
        });
        

        browser.executeJavaScript("exportKeyStat2CSV()");
        
        try {
			Thread.sleep(3000);
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	
}
@Thailer86

This comment has been minimized.

Copy link

commented Apr 18, 2019

After some reverse engineering I found out, what the "security" is to download the CSV as before. They introduced a new HTTP request header attribute called "Referer" where you have to provide a value (the URL from where you clicked on the button).
Using my Java client, this works perfect:

HttpURLConnection.setRequestProperty("Referer", "http://financials.morningstar.com/income-statement/is.html?t=AAPL&region=usa&culture=en-US");

@G4rpez

This comment has been minimized.

Copy link

commented Apr 20, 2019

@Thailer86 U R GREAT!!! my java client works again with your help. Thanks a lot

@Lualobus

This comment has been minimized.

Copy link

commented Apr 25, 2019

Somebody has been able to fix it on Google SpreadSheet?

=importdata("http://financials.morningstar.es/ajax/exportKR2CSV.html?t=US0846707026")

@dehaosun

This comment has been minimized.

Copy link

commented Apr 26, 2019

@Thailer86 Your are awesome~ it works.

@msulwa

This comment has been minimized.

Copy link

commented Apr 29, 2019

Thank you @Thailer86

@megahoy

This comment has been minimized.

Copy link

commented Apr 30, 2019

@Thailer86
thanks man!

@megahoy

This comment has been minimized.

Copy link

commented Apr 30, 2019

Sometimes i think - where is morningstar take data for their financial reports: Income Statement, Balance Sheet, Cash Flow? From SEC - https://www.sec.gov/edgar/searchedgar/companysearch.html or maybe directly from Edgar https://www.edgar-online.com/ ? And if this info is public - then maybe we should use SEC site instead of morningstar? And the problem is how to get those data in JSON or XML format from SEC? I have a lot of questions - but i don't have answers. And it would be great to stop using morningstar and to start using the origin financial sources like SEC or Edgar.

I found how to get annual report of Apple in Excel format directly from SEC - https://www.sec.gov/Archives/edgar/data/320193/000032019318000145/Financial_Report.xlsx or you can click the link 'View Excel Document' on this page https://www.sec.gov/cgi-bin/viewer?action=view&cik=320193&accession_number=0000320193-18-000145&xbrl_type=v to get the same report.

@Sheperd2019

This comment has been minimized.

Copy link

commented May 1, 2019

@Thailer86

Hi I am not sure what works again. I was using a google spreadsheet that was automatically scraping data from morningstar. Will that spreadsheet work again? Really searching for a solution but I am not an expert at all.

Thx in advance.

@cesarioalmeida

This comment has been minimized.

Copy link

commented May 2, 2019

Thanks @Thailer86

@megahoy

This comment has been minimized.

Copy link

commented May 4, 2019

I am using this request to get data:

GET /ajax/exportKR2CSV.html?t=AAPL HTTP/1.1
Host: financials.morningstar.com
Referer: http://financials.morningstar.com/income-statement/is.html?t=AAPL&region=usa&culture=en-US
User-Agent: PostmanRuntime/7.11.0
Accept: */*
Cache-Control: no-cache
Host: financials.morningstar.com
cookie: JSESSIONID=23DF3FB1A9B0884DBF83746ABC26F053
accept-encoding: gzip, deflate
Connection: keep-alive
cache-control: no-cache

You should always add Referer header, because otherwise request will fail.

@Sheperd2019

This comment has been minimized.

Copy link

commented May 5, 2019

@megahoy
Hi can you please tell my where this solution is used for? I bought an automatic google spreadsheet that retreives data from morningstar, but that stopped working. Is your suggested solution a way to get it working again?

@Peet005

This comment has been minimized.

Copy link

commented May 6, 2019

Anyone an idea how I should change my Excel VBA code to get this "Referer" into my code?

Current code:

ConnString = "TEXT;" & “http://financials.morningstar.com/ajax/exportKR2CSV.html?t=KO&denominatorView=raw&number=3"

With ActiveSheet.QueryTables.Add(Connection:=ConnString, Destination:=Range("A1"))
.Name = _
" "
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = False
.RefreshPeriod = 0
.TextFilePromptOnRefresh = False
.TextFilePlatform = 775
.TextFileStartRow = 1
.TextFileParseType = xlDelimited
.TextFileTextQualifier = Flase
.TextFileConsecutiveDelimiter = False
.TextFileTabDelimiter = False
.TextFileSemicolonDelimiter = False
.TextFileCommaDelimiter = False
.TextFileSpaceDelimiter = False
.TextFileColumnDataTypes = Array(1)
.TextFileDecimalSeparator = ","
.TextFileTrailingMinusNumbers = True
.Refresh BackgroundQuery:=False
End With

@toanong

This comment has been minimized.

Copy link

commented May 22, 2019

@Thailer86 Thanks for the tips. I tried to implement this in Python 3 but it didn't work. Any ideas?

import requests
import pandas as pd
from io import StringIO

url = 'http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=AAPL&reportType=is&period=3&dataType=A&order=asc&columnYear=2019&number=3'
hdr = {'Referer': 'http://financials.morningstar.com/income-statement/is.html?t=AAPL&region=usa&culture=en-US'}
req = requests.get(url, headers=hdr)
data = StringIO(req.text)
print(pd.read_csv(data))
@toanong

This comment has been minimized.

Copy link

commented May 22, 2019

Actually, it worked. I messed up the ColumnYear which should be 5 instead of 2019. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.