Hacker News Reader

Comment by yourapostasy | Hacker News Reader

Why isn't this data out on torrents? Is it somehow copyrighted?

There are a few reasons.

1. Almost all buyers are institutional funds or individuals with the means to trade as well-informed investors. To put it succinctly, they're in it as serious business, because it's incredibly expensive. They have no incentive to make the edge they just purchased for five - six figures public.

2. These vendors go to various lengths to protect the data, including steganographic "trap streets" to identify the account the data belongs to.

3. There are significant legal liabilities inherent to releasing the data without a redistribution license. You agree to these explicitly by buying the data.

Sorry, I should have clarified my question. Why aren't the exchanges themselves publishing the data onto torrents?

My layman's understanding is the more actors that participate on the exchanges, the more volume, the more the exchanges profit. If this is true, then making the historical trading data would encourage more entrants onto the exchanges, and the exchanges would earn far more than selling the data to a few large institutions.

I think it's not like there are tons of qualified people with a great interest in quant-y things whose only barrier to entry is a lack of data. It's sort of a monopoly+monopsony thing (I guess, this is highly likely wrong) where there's no incentive to expanding sales further because you're already hitting almost the entire market, and entry to the market isn't a problem for potential buyers.

Actually, Nasdaq makes more money on licensing its data than on the rest of its businesses. You can go to each exchange and buy the historical data directly. Generally, this is better for backdating depth of book data than say a Bloomberg in my experience but requires a lot more upfront work (parsing, normalizing). Additionally, any non-hft trader that handles institutional trading needs to pay a small fee (roughly $20/month per exchange) in each system that requires real-time data. Also, little-known fact, no retail trading orders hit the actual exchange. Even institutional investors who are not brokers (like most hedge funds) must execute through a broker/EMS.

I have a hard time seeing how it could be copyrighted, at least if we are talking about comprehensive stock listings organized in the obvious way (e.g., a table of prices organized by date, where the stocks included are chosen by some straightforward criteria).

In the United States I'd expect this to be covered by Feist Publications, Inc., v. Rural Telephone Service Co., 499 U.S. 340 (1991).

[1] https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R....

[2] https://www.law.cornell.edu/copyright/cases/499_US_340.htm

It's considered a "compilation". Feist addresses this in the links you supplied. However, if you're not used to reading law, this might help explain it in normal English. (Court documents are most definitely not normal English!)

https://www.unc.edu/courses/2006spring/law/357c/001/projects...

That was why I threw in the mentions of comprehensiveness, obvious organization, and obvious selection. Feist requires at least a minimum degree of creativity in the organization and selection for a compilation to be copyrightable.

A compilation whose selection criteria is, for example, "everything traded on NASDAQ" and whose organization is "order by date" seems unlikely to me to have sufficient creativity to qualify.

Nondisclosure agreements.

Reply on news.ycombinator.com