Ask HN: A stock trading platform for hackers? | Hacker News Reader

Ask HN: A stock trading platform for hackers?

25 pointsjashmenn21 comments

There seems to be a lot of talk on HN about HFT, short-selling, and the stock market in general. Using AI / Machine Learning / Data Mining allows smart hackers like us to more-or-less print money (according to the newspapers anyway).

Okay, I realize that I probably won't retire off of this, but data-driven stock trading just seems like a lot of _fun_. I'd like to try my hand at it even if I lose my "investment".

This immediately brings me to a practical problem: what is the best way to make a (HTTP?) request over the internet that buys a stock? I've know that some sites have APIs, but I want to know what you recommend.

My question two-fold:

1. Does anyone know of a good trading platform for (non-windows) hackers? 2. Is HFT (or anything close) possible for an individual hacker without a huge cash position?

I'm a total noob in this area, so please feel free to post any recommended resources as well.

View on HN

I was previously a professional trader.

For quantitative mid frequency trading on a retail size account that doesn't require ultralow latency: Many pros use IB personally and their java API is quite good. You'll be disappointed to know that MOST finance apps are built for windows environments and are frustrating to work with. IB's is quite reasonable. IB also lets you pull historical data that is quite decent, though they cap the rate at which you can pull historical data bars.

http://www.maxdama.com/ has written quite a bit about the IB API and I've found the provided documentation is actually quite good.

For pseudo HF type trading I'd say to take a look at lime brokerage.

REDI plus (Goldman sachs' platform) also has reasonable resources, though its harder to set up an account. for IB I think you just need 8k or so to open an account. Transaction costs are also the best for most retail/hackerish type applications (smallish quantitative based trading).

Also i'm interested in collaborating with HN people regarding trading strategies. I think the HN crowd is actually sort of the idea crowd to try to make it. There seem to be quite a few pros that actually read HN as well.

Would love to get in touch with anyone who is interested in working on strategies or even just talking.

More on IB: so you basically just have to fund an account with whatever their minimum is now (might be 10k now).

then to get most US stock/options/futures data, it only costs 10 bucks a month for their live feed. If you make trades, your commissions go towards paying down your data fee. If you don't trade for a few months, consider it 10 bucks a month for the opportunity to develop strategies in a real environment.

Also, they give you a "paper trading acct" which behaves like a real account, except it doesn't post real trades. you can forwardtest your strategies on live market data with this.

IMO, this is probably the best way for HN people to get involved.

I also realize that it probably sounds like I'm selling for IB, but i'm not. Just a happy customer for 4 years.

Interesting! You seem to know a lot about it, are there any books on stocktrading generally and on algorithms and/or computerized trading specifically, that you can recommend?

As an individual over even the fastest internet connection there is no way you can do High Frequency Trading. You need latencies of sub-millisecond to do this. The guys that do do this have very expensive hardware co-located at the exchanges. All this costs a LOT of money (a friend of mine who developed an HFT platform for a large bank reckoned you'd need about a million bucks to have even a vague chance of comepeting). Additionally it's also VERY risky, and not the free lunch it's portrayed in the media. You're "picking up pennies in front of the steam-roller" - one mistake and you're f*cked. And trust me, mistakes get made: if an order doesn't fill/complete and you're left with an outright position you can lose a lot of money very rapidly.

So forget HFT. However yes there may be opportunites for relative value / alpha / rich-cheap type analyses, which AI/ML could help you with. But bare in mind that markets are often "confounding" - and never do what they are supposed to. So play carefully.

Edit: I should also add that HFT requires a very fundamental knowledge/understanding of contracts and their pricing (mostly futures), how exchanges match and fill orders, how curves are constructed, how the markets operate and so on.

Agreed. As a developer in the field, I confirm access is very expensive and the software base deployed is very heavy (custom protocols built over TCP and engines). Basically, if you go through HTTP (efficiency put aside) then you go through a broker. This adds up latency and voids any chance of HFT.

> You're "picking up pennies in front of the steam-roller" - one mistake and you're f*cked.

Isn't that why exchanges have a simulation FIX ports?

Depends. We use simulation to validate connectivity and business logic. No link to the potential future bad decisions taken during trading.

I have been dabbling in this and it is pretty interesting. It is a huge time sink by the way. Also, you are playing the odds and you need a certain amount of cash or it will be gone before you get anywhere. If you end up trading frequently ("pattern day trading") you will be required to keep a higher balance in your account, BTW.

You might start with ninjatrader, which you can get for free. They can connect to yahoo for free price quotes (daily only), and also to some brokers for real time quotes during the day, if you have an account (I use TDAmeritrade and have been pretty happy with it).

ninjatrader has a (wacky) programming language and you can code up the conditions for buying and selling. The really neat thing is that you can run your "strategy" against historical data to see how it would have worked, and tweak it, etc.

Overall it is really painful to work with (if you are a programmer). But it does allow you to learn a lot without spending or risking any money until you think you are ready.

I believe TDAmeritrade has some kind of API you can call directly, but you have to contact them about it. Another one that I have looked at is Interactive Brokers. I haven't used them but I think they are well regarded. They are really aiming at professionals ("industrial" vs. "retail"). Possibly more what you're looking for, since they have a download page.

I would hold my nose and spend some time with ninjatrader (or something like it) just to see what's involved. You can code up your strategy and see on a chart where it would have you buy and sell, and how much you would gain or lose.

It's really pretty tricky and you should have some feel for what works and what doesn't (and when it works and when it doesn't) before you go all in.

Do you know if there is a place to download historical "real time" data for testing?

By "real time" presumably you mean tick data. This is generally not available for most contracts on most exchnages unless you are prepared to stump up a fair wedge of cash. However, i did once find on some russian website per-second historical data for various currency pairs - although I forget the site now.

However depending on your trading strategy I doubt you'll need tick data. You may be able to find per-5-minute data for free on some stuff. And EOD is generally free for most contracts.

I have also pulled 1 second bars for currency from IB. they throttle your query rate though, so it can take some time to get your data.

Tick databases large enough to do backfill testing on (a few years worth) are usually a few TB for equities markets.

In my experience, between 4-7GB compressed per day for top level regional data in US equity markets.

http://datashop.deutsche-boerse.com Tick data: Price, volume, time, total volume for every options series and every futures contract - 120,00 Euro per calendar month.

You would probably be more interested in top of level order book data though, thats 300,00 Euro per calendar month.

Don't know of other exchanges, but seeing as the FESX is one of the most liquid products in the world it should be sufficient for testing. Check the sample.

IB will give you up to 1 second resolution (open, high, low, close) for most US equity names.

getting bid ask tick data is harder/more expensive to come by.

I use IB, and it's the best I've found so far.

My dream, however, is to be able to trade on a headless linux box using REST. Currently, I have to fire up my windows PC every morning before the market opens, connect to IB, and start my algorithm running. Needless to say, if I go to that much trouble, I want a trade or two to occur that day, and I think it influences the algorithms I write.

It would be better to have something on a dedicated Slicehost slice that updates every minute or every five minutes, and waits for some event that may only occur once a month. I don't need tick data, and I'm not sure I need even one second data. Actually, the data feed is not the issue. The issue is that I have so far been unable to find an API that will allow me to initiate trades on a headless linux box.

A few brokers offer APis such as TDAmeritrade, and Interactive Brokers. IB has 10k minimum opening (though I believe you can transfer some of that out after).

More importantly to be a pattern day-trader and get the 4-1 margin you have to maintain a minimum account of 25k.

Also you should check out marketcetera http://www.marketcetera.com/site/ - though I don't have first hand experience with it.

To your second question - no not really. HFT works by scalping a stock for pennies - buying and selling (tens/hundreds) of thousands of shares for a very short timeframe (seconds, minutes, hours?) (Tangentially look into the 'sharpe ratio' on wikipedia)

I don't think you can compete in the HFT arena - I would personally recommend looking at medium-term strategies and possibly using options for leverage.

I've been thinking about building a simple fundamentals strategy. More interested in EOD data and indicative data, though.

HFT itself is just a good way to have your money taken away by smarter people... but long-term alpha is probably still doable.

If you're into forex/CFD/futures rather than stocks, you might want to take a look at Metatrader. It's free, it is very widely used and it comes with its own C-based programming language, MQL4. I've used the language to set up my own custom indicators and expert systems (for real-time automated trading). I recommend this as a very hands-on way to learn technical trading.

http://www.metaquotes.net/ http://www.mql4.com/

Forgot to mention: algodeal.com

Never tried it myself and can't vouch for them, but it seems to be an interesting concept and might let a lot of hackers get their feet wet without any monetary investment.

http://www.milleniumit.com/media_room/gui-press_releases.php offers turnkey HFT solution.

    HFT = Algor trading + Flash trading

and Flash trading is possible if you've a seat in the exchange.

http://www.counterpunch.org/whitney04162010.html

Reply on news.ycombinator.com