Advice on the implementation of a real-time trading system [closed]
stackoverflow.com
stackoverflow.com
He is not a broker, you can't really interact with any exchange directly. He needs to use some bank (which will already do 99% of what he is implementing).
Without knowing exactly how the market systems work, this is a toy, he should have talked to someone BEFORE writing any code. Things are way more complex and simpler than it looks. It's extremely simple to use bank services to do that for you and close to impossible for a developer to just jumpt into writing trading software without a bank backing it.
http://news.ycombinator.com/item?id=5041040
"Markets too complex for high frequency trading software"
"Financials" sucks and they hide every inner workings...
The backtester is designed to also be a trading engine. You feed zipline data in "events" where each event is a point in time. Zipline doesn't care if the events are live or replayed from a database - it processes the data and executes any trades in the order book.
The tricky part here is that zipline is more aimed at implementing trading algorithms. You'd have to write a simple algorithm to implement the buy/sell orders coming in.
What I ended up doing is to cross-post the questions in some mailing list/groups that are slightly related to the topic (celluloid, any active ruby/my-lang group).
That's a good rule. It means SE can focus on that thing. But it's a great shame that the huge knowledge of the SE sites is locked in that format.
There's quite a lot of heat generated around the fuzzy borders of acceptable and unacceptable borders, and having a question closed or deleted doesn't feel good.
Meta is death. I kind of agree, yet SE does have active meta. SE also has chat stuff.
So, would it really kill them to have "freeform SE question site"? It could be a paid feature (although recommending people buy a sub to post a question would cause all kinds of furore).
Thus, you'd have:
http://gamedev.stackexchange.com/
http://meta.gamedev.stackexchange.com/
*http://freeform.gamedev.stackexchange.com/
It would still have strict rules, and mods, and etc. But it would allow more open form discussion. They might need to have a different structure to allow threading and better replies?
Because of this, http://programmers.stackexchange.com exists, which is a site that allows more free-form questions.
May be start a SO type site which focusses specifically on Artchitectural how-tos ? Like "How to design a real time trading system", "how to design a supply/chain software" etc. What do you think ?
This small toolset could also elaborate various business models and business processes as well.
https://groups.google.com/d/topic/celluloid-ruby/9RHPmYkxGtU...
https://groups.google.com/d/topic/rubysur/ZJoZxDVcRYU/discus...
Celluloid looks interesting have you played with it yet?
I know little about stock markets but I quite like the idea of spawning workers to monitor orders after being placed. These could then update a page live to the customer.
But now that it's here it would be great to have some comments from HN though ;)
How long do the orders need to be monitored for? Indefinitely? If so, is it really feasible to keep them all running? How much data do they need to analyse / aggregate to make their decisions?
Very interesting question.
In my case "decision making" is only based on some fixed price threshold in order to provide some risk management. So the amount of data to aggregate would be minimal. But the data to analyse came directly (as a huge stream) from the markets.
* Actually the previous statement could be wrong: You might also want to do this on a per order basis, although they are on some sense functions of each other.
I would suggest the following:
For any market which you have a position in, subscribe to that market's price feed. I'd use zeromq for this, but there are plently of pub/sub libraries and products. The previous point implies that you are republishing your market data feed into specific channels. How they are mapped depends on many factors, but basically you want to de-multiplex the market data so that you are able to pick out one market's price data as a stream of updates from the whole.
Once you have a feed of the market with a position you can use it to transform the stream of market data to multiple streams of values. You might have new steam of PnL values for each order you have on the market, or a stream of price differences or whatever, you then re-publish those streams. Depending on how many of these derivative streams you have and the rate of data, you might want to do some downsampling too.
What I've described above is pretty similar to the market data architecture in hedge funds that I've worked at. Have a look at the ideas behind function reactive programming http://en.wikipedia.org/wiki/Functional_reactive_programming for more info on stream processing.
For the front end, you'll bridge the particular stream of data to a WebSocket (probably translating it to json or a binary data format that javascript can understand at the same time). I don't know how you do that in RoR, I'd use gevent or eventlet in the python world. You'll then need to route this data in some way to a representation of the order/position in the front end. I'd suggest looking at knockout.js, ember.js, angular or something similar as those declarative style libraries will massively reduce the complexity of your client side coding.
Last bit is visualization and there's really only one sane choice for this imho: D3 and libraries built on top of it. Look out for http://square.github.com/cubism/ and http://square.github.com/crossfilter/ and http://code.shutterstock.com/rickshaw/ for those bits.
Bit of a whirlwind here, hope it helps!
Edit: Here is a small tutorial: http://bjorngylling.com/2011-04-13/postgres-listen-notify-wi...
Keep the database for storing stuff that you don't need in ram. Keep the rest in ram and message queuing/pubsub bus architecture.
Plus events in a database engine are a pain in the butt to write tests against.
TBH when I write java, I usually write virtually no code: I just glue other shit together.
I don't get why people have so much against Java - it gets shit done in incredibly short amounts of time.
I find its a great language for integration in general and talking to "legacy enterprise" systems.
I do find it verbose, and have other remarks about it, but it does the job well enough.
Thanks
Trading software has the bureaucratic aspect (IO, etc) and the really money making aspect (calculations).
Let me show an example how f'up pretty much all non-functional languages are in solving SIMPLE interest (total cost of operation) calculations:
https://github.com/eduardordm/cet-java/blob/master/src/main/...
This can be solved with a few lines of OCaml - not that current O(nˆ2) approximation crap.
Of course everybody is free to act on his own karma (in a few countries) - but you will have a much better life by doing some more useful work that will benefit all human beeings, not only a few criminals that build weapons of mass murder just for the mentally poor reason of making more and more money.
Programmers haz powers to push human evolution, don´t waste your life on golden lies, that will turn into deadly dust in the end!
http://www.glassdoor.com/Salaries/quantitative-analyst-salar...
Programming getting into high frequency trading is "high risk, higher reward". It would be better to use our Neural Networks to find risk factors for prostate cancer you'll get in 25 years, but the money just isn't there. AKA high risk, low (possibly no) reward.
I'm under no illusions that I'm doing God's Work, but if the bar for useful work is something that will end "war, disease, crime, militarism, xenophopia, lack of education", I think >99% of us fall quite short of it. Maybe I fall shorter than you do (you don't mention what it is that you do), but I still enjoy my job.