Tell HN: Yahoo Finance has apparently killed its API | Hacker News Reader

Tell HN: Yahoo Finance has apparently killed its API

235 pointsphp2011100 comments

Yahoo Finance has apparently killed is API. Zero warning. Lots of apps probably use this.

Before, you could get stock information by using http://download.finance.yahoo.com/d/quotes.csv

Now, you get the following message: It has come to our attention that this service is being used in violation of the Yahoo Terms of Service. As such, the service is being discontinued. For all future markets and equities data research, please refer to finance.yahoo.com.

What violation of TOS? People have been using this for years without any issues.

If you are going to cut this off, how about a warning and heads up?

Guess that's what we should expect from OATH / Verizon.

View on HN

IEX has an official, documented, supported, relatively clean, and entirely free API for their market data:

https://iextrading.com/developer/docs/

It's just US stocks, and quote data is just for their exchange (and you can get the deep book if you're interested in that stuff). But they have basic company info: earnings estimates, split info, dividend info, etc. as well.

IEX here. Thanks for mentioning our API. We are excited to see what everyone will build. Just to reiterate, the API is free for personal and commercial use, doesn't require an API key, and is reliable enough to drive our own applications.

Real time quotes and trades are available from our own market data, but we also provide market wide prices (SIP) on 15 minute delay in addition to the company info mentioned above.

We are very actively developing the API, and will continue to improve data sets, features, and performance. If you experience any issues we have community support at https://github.com/iexg/IEX-API/issues which we actively monitor.

Thanks for building this! I noticed that the charting api have EOD prices for past 5 years, is there a plan to support longer durations?

We are currently working on new datasets including EOD data for the past 20 years. Our market data is available now through REST and WebSockets, but we are working on adding more data via WebSocket, and introducing HTTP streaming in the near future.

Yes, IEX only. But still great service, great tools. All for free. I mean how often do you see websocket support?

If you want or need high res quotes from all exchanges including options data, you can also take a look at Nanex

http://www.nanex.net/

And thanks to everyone for posting great links ;)

What is the pricing like?

It is free (IEX).

They broke functionality on their own pages, yikes.

E.g. the export button on the desktop version of https://finance.yahoo.com/quotes/MMM,IBM/view/v1?ql=1

You'd think after you take down an API you've be furiously checking the server logs for any referrers coming from inside the house.

Half dead company running on fumes. They probably don't even know half of what they run, let alone have staff to check the logs.

"Half dead company running on fumes"

And for good reason. I don't go to yahoo for its clever articles or its mail or its search function ... I go to download currency commodity and stock information.

At least they could have explored offering the million people like me the info for a couple of bucks a month ... 2 million a month is worth looking at.

Instead a cryptic adios ... its brand suicide (or murder seeing how verizon owns them now)

Or they just want to get out of that business.

Yeah, if someone gives a shit. I would say that has not described Yahoo for some time now.

There are lots of talented Yahoo employees who read HN, and cringe at what teams on the other side of campus are doing.

Recursive Yahoo TOS violation

There's a thread about this on Yahoo!s own forums:

https://forums.yahoo.net/t5/Yahoo-Finance-help/http-download...

I am incredibly amused by the person who refers to himself as a customer of this. If you want to be treated like a customer, find someone who will charge you money in exchange for the service provided.

The last post on another Yahoo! forum[1] is from an admin that says:

The new download issue which began 11/01/17 and returning an error 999 for most users is currently being investigated and we hope to have it resolved soon

This was before the TOS violation response was being returned so I'm assuming not everyone in Yahoo! was prepared to deal with this.

[1] https://forums.yahoo.net/t5/Yahoo-Finance-help/Is-Yahoo-Fina...

Sorry in this day and age using a product makes you a customer and the product. Facebook, Google considers you one.

There's obviously way more nuance to this though. APIs of this nature rarely charge a fee.

You mean APIs that provide market data? Quite the opposite.

Intrinio (https://intrinio.com/) has an awesome API for Financial Data. While not completely free, they do have 6 months free program for developers.

They've also been featured on Indie Hackers: https://www.indiehackers.com/businesses/intrinio

I use Alpha Vantage, just for a personal (1 user, me) application. The service is free, you just need an API key. - https://www.alphavantage.co/

https://www.alphavantage.co/support/#api-key "While we are proud to provide free API service with no daily/weekly/monthly call limits, we recommend that API call frequency does not extend far beyond ~100 calls per minute so that we can continue to deliver the optimal server-side performance. If you would like to target a much larger API call volume, please drop us a note..."

Implementing this to replace a very low usage exchange rate form. The format of their data is... odd. E.g. -

{ "Realtime Currency Exchange Rate": { "1. From_Currency Code": "BND", "2. From_Currency Name": "Brunei Dollar", "3. To_Currency Code": "USD", "4. To_Currency Name": "United States Dollar", "5. Exchange Rate": "0.73326846", "6. Last Refreshed": "2017-11-03 17:37:12", "7. Time Zone": "UTC" } }

Why the numbers and mix of underscores and spaces in keys?

I don't know why they did that, you would need to ask them. They seem to be open to questions and suggestions. My guess is that it's historical and now it would annoy too many users to change.

I remember seeing this API from another post.... Any idea who's behind it?

If you built a business around data that you didn't explicitly have the rights to you built that house of cards yourself.

You realized that’s probably half of the startups and companies out there right?

That doesn't make it any less true.

Businesses can fail. That is true as well.

I guess we should never build businesses.

That's a bit reductionist.

I'm saying we should enter into formal contracts for services that are vital to the running of our businesses. That way there's more protection against getting the rug pulled out from underneath us.

That's easier said than done. Try getting a meeting with the VP of Facebook, who's bonus depends on increasing ad revenue for Fortune 500's with a $100 million dollar budget. Or plow through and take the risk, and be the next Buffer.

Thanks to the MVP religion the expectation is now set that for any business should be able to get $2m in revenue with a $5,000 investment from your uncles and cousins. More often than not you’re taking these risks because you don’t have the money or infrastructure to ‘do it propperly’.

The risks you take in your business should be calculated risks. 100% of your business on data you haven't somehow ensured access to? Bad calculation if you ask me.

Most startups and companies (try to) act in good faith and will acquire a license. And, if they did not, they should foresee the legal and operational consequences.

Well YouTube was similar when it first started, no? People were uploading whatever they wanted, and eventually they rolled out DMCA takedown and reporting features for copyright infringement violations.

Over time, the service was setup with a full feature set to handle all of this, but it didn't exist when YouTube first started.

I mean just because YouTube broke the rules and made it doesn’t mean everyone else will.

If you believe that, then... Wanna buy a lottery ticket from me?

Sometimes a house of cards is exactly what you need to prove that it is a good idea. Then you can look for weaknesses and fortify the structure.

You just can't blame the wind when it comes and knocks down the house.

Also, I think half of startups and companies is a little bit high. I can't imagine that the majority of large companies are cool with this kind of risk.

That data is expensive. Do things the right way, or why not outsource your programmers to India while you’re at it then?

Reminds me of all the Twitter API issues that clients face.

That sounds like Google Books.

Not according to the courts.

There API is still working via http://finance.yahoo.com/webservice/v1/symbols/allcurrencies...

have you got any further documentation on hwo to use that? thank you.

Documentation... Oh, my sides!

Thanks for this

At TagniFi we have a free account that provides access to end-of-day market data (TagniFi Markets) via our API at no cost. Data items include high, low, open, close and volume along with adjusted closing price (for splits and dividends). Documentation is at http://docs.tagnifi.com/article/63-search-tagnifi-markets. Sign up at https://www.tagnifi.com/trial

Note: we only limit our trial to the Dow 30 for our fundamental data (balance sheet, income statement, cash flow statement). Market data is not restricted to the Dow 30 for the trial which does not expire.

Quite happy about the suggestions for similar APIs here. They’re somehow hard to find in the forest of overpriced professional services.

I‘m using a different Yahoo endpoint with ticker.sh[1], which is still functioning at this time. Maybe there‘s soon the need to find yet another alternative after the Google endpoint stopped working recently.

[1] https://github.com/pstadler/ticker.sh

Yeah. It's been on & off for a while but yesterday looks like it went for good.

I had to fix my script switching to:

http://www.bloomberg.com/markets/api/quote-page/{CURRENCY_PA...

is it documented somewhere or you just analyzed your browser requests?

You'll get IP banned if you go beyond a certain threshold, but looks like you can find other API endpoints by looking at browser requests.

The API at https://query2.finance.yahoo.com seems to still work [0]

[0] https://query2.finance.yahoo.com/v8/finance/chart/VZ?period1...

Thanks but this is only returning historical stock price data while the yahoo download quotes.csv returned stock market data including ebitda, market cap, price ratios and much more.

> What violation of TOS?

It is against Yahoo's TOS to scrape the data there. So yes its darn entitled to even expect a heads up.

You don't "scrape" a .csv.

It is what it is.

Downloading a csv isn’t scraping.

I don't know if the csv was meant to be an api or just giving the option to someone visiting yahoo finance (and therefore viewing ads) the ability to download data they viewed. If you go straight to the data and bypass the ads, you are sort of scraping.

Well the geniuses at Yahoo also broke their own site--if you go to Yahoo Finance and try use their own download link it takes you to the same message. So they must not actually care about the ads.

You must be joking. It’s an API, ads are irrelevant.

https://polygon.io - real-time stock data APIs

Roughly 10 years ago I build an Excel stock screener using this API. The data quality was always terrible.

I wouldn't trade on it but they were reasonably good to get a sense for where stuff trades. E.g. monitoring your portfolio with this worked well, esp for EOD data.

Intrinio (http://intrinio.com/) currently offers the least expensive Real-Time and REST API for US stock prices.

You can access it via WebSocket, or Web API. No other firm currently offers this data via WebSocket which is especially helpful as you only need a few lines of code to implement. Additionally, Intrinio's customer service chat support is live practically 24/7.

The real-time feeds can be found here: https://intrinio.com/marketplace/data#?category=prices&sub_c...

The REST API for prices would be best in this feed: US Public Company Financials (https://intrinio.com/data/company-financials)

This blog will tell you everything you need to know to get started using the real-time API: http://blog.intrinio.com/stock-prices-from-intrinio/

It's not the least expensive. https://www.alphavantage.co/ is free.

I interviewed the people behind Intrinio[1] for Indie Hackers. I haven't used their product myself, so I can't tell you how it stacks up against the Yahoo and Google APIs that were killed this month. But I do know that they provide financial data and APIs to developers, and their primary goal is making it super cheap and pleasant to use.

[1] https://intrinio.com

I wrote a small cli/library to pull tickers and historical quotes from various sources directly to csv or json. It still works with the new Yahoo format, but I wouldn't count on it sticking around. Also pulls bitcoin data from gdax.

https://github.com/markcheno/go-quote

The undocumented Google finance API was also turned off this month after many years.

There's a severe dearth of good, free, market data API's.

If both shut down within a short period of time it's probably due to the vendor. I'd guess one of the major exchanges in the US has changed the terms of the license or started enforcing it. They make money with this data so it's understandable that they don't like free APIs.

Likely the NYSE, they've decided to make some money off of algorithmic trading by charging a non-display fee, where it costs thousands of dollars per month to use stock quotes for anything other than displaying to a human, and have been on a warpath enforcing it, causing a large fight between the NYSE and Bloomberg, Citadel, etc. with lots of lawyers and government regulators involved.

I can't imagine they're happy about people possibly using Google or Yahoo Finance APIs for similar purposes either (since most of the uses of market data APIs are probably for non-display use).

http://www.businessinsider.com/bloomberg-letter-targets-nyse...

https://www.nyse.com/publicdocs/nyse/data/Policy-NonDisplay_...

https://www.quandl.com/search

I had been downloading about data for about 30 stock symbols once or twice a day for years.

This was a handy service that gave Yahoo credibility to me but evidently there were those who completely abused the handy service by downloading hundreds if not thousands of stocks minute by minute. It's those who wrecked it for their own need for greed. They should have been contracting with a service to provide them with data.

For the rest of us it is now gone. Yahoo could have easily given each person perhaps a limit of 10 or 100 downloads per day by counting the IP addresses as other web sites do.

So don't blame Yahoo or Verizon. Blame it on those that have to have the world for free and will abuse whatever free service you provide. They are the ones who are all about themself who wreck the world for everyone else.

I did notice that in DDG, "!finance TICKER" now goes to other sites, where recently it used to go to yahoo finance. I wonder if the change was due to this, and if DDG had any warning?

I'm writing an article about this... Questions to anybody that knows:

1. it looks like this was just a CSV download. Was any form of authentication (even just a Yahoo login) required to download the CSV? I'm just trying to figure out how, if Yahoo actually thought of issuing a warning, it would have known who to send it too or how to get news of the shutdown circulated to the right people.

2. If it was a CSV download, how up to date was the data?

3. Does anybody have a sample URI for a filtered version of this CSV request. For example, just two symbols?

Thanks

David Berlind ProgrammableWeb

here's some info on how it worked: http://www.jarloo.com/yahoo_finance/

Was there ever a TOS for this API? I was under the impression that this was just an internal API used by Yahoo Finance itself, and that they've never marketed it in any way.

Not just finance.yahoo.com, but also access through query.yahooapis.com.

https://query.yahooapis.com/v1/public/yql?callback=&diagnost...

Zero warnings? Didn't Yahoo die like a few years ago?

Globally about a billion people annually use at least one Yahoo service.

The answer to your question is: nope.

A non profit that I support has been relying on Yahoo for exchange rate information for nearly 10 years. I knew when I inherited the app that this was imminent, so it wasn’t too surprising.

There are plenty of alternative paid APIs out there that made switching easy. Hopefully others who had legacy developers integrate with the financial APIs are able to switch easily too.

Good time to remind everyone that Google Finance said they would delete everybody's portfolio's in November and if you want the data you should download it.

You don't see this unless you go to the Portfolios page: (not my screenshot) https://imgur.com/0o4Ehxe

One more existing alternative: https://developers.exante.eu

It's broker company itself and they have API with EOD and realtime data on stocks and derivatives for many exchanges including US, Europe and Asia.

You can even trade with that API if you want (guess it won't for free though).

Google lets you import stock price info into Google sheets using the =GOOGLEFINANCE() function.

But for more than a year, Google's mutual fund prices have been showing day old information many hours after the market closed.

So i had been using the Yahoo API to populate an investment portfolio spreadsheet.

So now both Yahoo and Google don't work. You get what you pay for.

There are a lot of good free alternatives on the comments but I'm curious how reliable they are.

Obviously, Xignite is the leader in paid service but it's way too expensive for startups (approximately 20k for the cheapest option).

IEX and AlphaVantage have decent customer support so I trust them, but I'd rather have a cheaper, paid service.

Intrinio (http://intrinio.com/) currently offers the least expensive Real-Time and REST API for US stock prices, AND their customer service chat support is live practically 24/7.

You can access it via WebSocket, or Web API. No other firm currently offers this data via WebSocket which is especially helpful as you only need a few lines of code to implement.

The real-time feeds can be found here: https://intrinio.com/marketplace/data#?category=prices&sub_c...

The REST API for prices would be best in this feed: US Public Company Financials https://intrinio.com/data/company-financials

This blog will tell you everything you need to know to get started using the real-time API: http://blog.intrinio.com/stock-prices-from-intrinio/

That API has been flakey since months ago. You can still get quotes from Yahoo Finance in other ways just fine though.

Just circling back here (hoping someone can help me with my coverage). Given how the endpoint required no authentication (no uid/pwd, key, or token), and therefore, Yahoo had no idea what developers were using it, how did you (speaking to developers here) expect to be notified of the API's retirement?

Earlier this year (~May) they shut down their API endpoint that rendered handy little stock chart images with a surprising amount of useful features. Any alternatives out there, other than tying together a charting lib and data API?

the robinhood API is unofficial (for now) but it's an easy way to pick up stock data for free.

https://github.com/sanko/Robinhood

For us non-financy-types, does anyone mind letting us know what this file looked like so we can, you know, potentially duplicate it and provide it to those who find it a necessity?

There are already a number of existing alternatives:

http://fixer.io

https://openexchangerates.org

https://www.exchangerate-api.com

http://www.xe.com/xecurrencydata

You find FX data everywhere because it's public. But Yahoo also had stock data. And that is hard to find for free online. I guess Yahoo had to shut it down because exchanges didn't like them giving data away for free via API.

The Apple STOCKS app on my iPhone is showing updates as I watch and says Yahoo! at the bottom. So, it seems Yahoo is still providing fairly real-time stock quotes via some means.

https://github.com/timkpaine/pyEX-zipline

its

edit - hackernews apparently doesn't value new ideas.

Please forgive my ignorance, but what possible intersection is there between blockchains (method of immutably storing small quantities of data in a distributed fashion currently limited by power waste, slowness, and needed storage amount) and providing real data about real stock markets to real people (getting data with proper license and providing it to others on request, currently limited by legal means)?

Coin exchanges are dealing with plenty of real money for plenty of real people and have real licenses and very real government oversight (at least in places like the US and EU) , so your snark is not very appropriate.
Regarding your actual point, I can't see the connection either.

Bitcoin futures on CME, coming soon. http://www.cmegroup.com/media-room/press-releases/2017/10/31...

If people not liking an idea means they don't like all ideas, how do you identify your bad ideas?

After 22 years they still can't handle a mailbox correctly, so that's not very astonishing.

Reply on news.ycombinator.com