Hacker News Reader

Comment by roel_v | Hacker News Reader

Set it up, trade 'virtually' (i.e., just simulate), prove your algos, get bankrolled by someone who is convinced of the algos worth. Quite straightforward. I'm highly skeptical of a hobbyist beating the competition though (maybe in micro niches, where one would add domain specific knowledge to get an edge - that's my pet idea on this topic :) )

View on HN

It is straightforward, but impractical. Getting good historical market data is very, very expensive. Even recording current market data is not cheap and not without significant difficulty.

True, but it's not impossible to get data sets to work from - on all quant forums the question comes up at least daily and the answers are always the same.

Why would one even consider entering these markets without the ability to demonstrate alpha over historical results?

Yes, your alpha needs to include the ability to beat transaction costs... so if you can't pay for the [relatively exceptionally cheap] cost of several years of your chosen market's historical data, maybe you're not really serious.

It's fun to play at being a financier, but let's not pretend that access to market data is the significant barrier to entry which is preventing disruption in the world financial markets.

The cost of market data and a system equipped to process it are not inconsiderable.

Historical tick data (quotes and trades) for US equities will run you $20k/year, minimum. US equities runs about 30-40GB/day so you are looking at almost 10TB/year. You need to be able to access the data quickly so you probably don't want to compress it and probably do want to duplicate the data, slicing and dicing it in different ways. Call it 20TB, minimum, per year of data.

Futures and options data will run another $20k/year, each, and futures has about as much data as equities and options has an order of magnitude more. That is another 100TB/year, minimum, per year, probably closer to 200TB.

How are going to to study these data? Even if you were able to operate at disk speed (which seems unlikely unless you have lots of disks and have duplicated and distributed your data amongst them) it would take you five minutes to run against one day's worth of data, or close to a day to run a year's simulation. If you consider more than a few cores in parallel, reading from disk is going to be seriously hampered.

None of this even considers the cost of the software necessary to support this effort: contending with corporate actions, tracking halts, etc.

This all adds up to a significant financial hurdle.

Surely though, one could pre-calculate potential future positions or other useful data on stock in advance of next day's trade between close and re-open of the markets?

he cost of market data and a system equipped to process it are not inconsiderable.

Not even close, if you consider the alpha for generating value. I know it's awesome on HN to think that generating 10M over 2 years entitles one to a "50M exit."... but that's not the way things work in finance.

20K is nothing... if one thinks it is than one is playing the wrong game.

And to be perfectly clear. I will take all of your action, every day, in prediction of market movements.. You tell me what you want. I will set a price and we will play for as much money as you like [up to my assets of 20MM or so.] I have no fear that I will beat you [or any one else] before you get me broke.

For a non-pro setup you're going to be accepting significant scope limitations in production - you're not going to be scanning the full real time feed across the whole market. Without colocation HFT isn't really an option either. So you can restrict your strategies to medium frequency against a small universe of assets which makes the technical side easier. Also the data side you won't need a full set of data, e.g. tickdata.com has data at $20/symbol/year (https://store.tickdata.com/prices?productMarketId=PRODUCT_MA...)

It's not really that simple. You can't really ever simulate your trading with 100% confidence since you're never entirely sure what your participation in the market will do to the results you see when you /aren't/ participating.

There is also plenty of anecdotal evidence that as you try and increase the amounts of capital you're putting to work in the market that it becomes tougher and tougher to maintain an edge in the markets.

Reply on news.ycombinator.com