We are moving into the quantitative finance industry
fpcomplete.com
fpcomplete.com
Standard Chartered traders use an excel to Haskell interface for risk management and trade execution.
S&P Capital IQ uses a Haskell DSL called Ermine for their reporting engine.
Tsuru capital is the largest hedge fund to use Haskell for trading but there many smaller shops like Alpha Heavy Industries.
http://www.haskell.org/haskellwiki/Haskell_in_industry
FP Complete repeatedly mentioned Standard Chartered's Excel to Haskell system as something that they would want to see more widely adopted.
And yes, R and Python are making real inroads in quant finance, but they still have a small market share. Most of the guys writing code either want MS Office integration (sadly all traders love Excel), raw speed (C/C++/Java), or easy prototyping (Matlab). People don't know Haskell, audit departments don't know Haskell, and Haskell isn't as fast as C++, so this is an uphill climb, but I wish them the best.
If you want to know the state of the art in quant finance, go stare at the horrible over-engineered debacle that is QuantLib and see what kind of programmers we have working in this area.
Why do you say 'sadly'? Excel is an incredible tool that other traders can work with, it is sufficiently expressive to run trades and backtests, and the connectors (e.g. Bloomberg) are better than the python equivalent
On an engineer pov with everything we invented (such as CI, or no single point of failure in databases) it is just purely unbelievable.
In a business sim class in college a couple of decades back, I discovered that the Lotus spreadsheets (as I said: a couple of decades back) had a totalling error which double-counted individual row totals in the bottom line (everything was twice as profitable as the spreadsheet indicated).
At an early gig, one of the senior developers instituted a practice of code walkthroughs on projects (only a subset of them). One of these involved, you guessed it, a spreadsheet (we used a number of other development tools for much of our work), in this case Excel. Again, numerous errors which substantively changed the outcome of the analysis. One of the walkthrough leader's observations was that you could replace all of the in-cell coding with a VBA macro making debugging far easier (all the code and data are separated and in one place each).
The particular analyst whose project this was: he insisted to the very end that this "wasn't a program" and he "wasn't a programmer" and that the walkthrough didn't apply to his situation. Despite the errors found and corrections made.
At the time (mid 1990s) the walkthrough lead turned up a paper from a researcher in Hawaii on the topic. I'm not certain it was Raymond Panko, but his 2008 paper (a revise of a 1998 work) discusses the matter in depth:
All that to say not that Haskell is going to beat Java/C++ in any visible time frame but instead to compare its use favorably with something like Python.
[1] See Paradise, Credit Suisse's Excel interop library (http://www.icfpconference.org/icfp2008/accepted/37.html)
[2] http://research.microsoft.com/en-us/um/people/simonpj/Papers...
I think the story still holds though: financial analysts are provided an upgrade path from Excel to Excel+Haskell backend, to Haskell embedded DSLs. The same sort of game might be happening with S&P in Boston.
This isn't that strange or unusual at all, the languages just sort of appear and get some local mass going, but nothing catches on globally.
(1) That quants want to program (or that the business should want them to) in something other than R, Matlap, NumPy, Julia, etc. (2) That the hard part of moving a model into production is translating the model. (3) That the model is the part of the system that will be the main driver of success.
Good Luck!
What drives success is having a good set of quants who can communicate with production capable software engineers. At the end of the day, the hard part of a quants job is usually finding concepts. Once they are found translating them into something actionable is usually reasonably easy.
That depends on the market, doesn't it? They didn't say they were explicitly building systems for Forex, right?
And who are you to tell people what they should "waste their brains on"? A lot of programmers I know in research labs/academia speak condescendingly towards really intelligent SV programmers who "waste their brains" on moving pixels around a screen in order to make people click more ads.
It has the regrettable feature that it strongly incentivizes some very smart people to spend their careers encouraging people to click on advertisements, and some other very smart people to spend their careers arranging for the profits from exploiting subtle equity mispricings to flow to one random set of rich people rather than another. But the net effect is pretty good, or more precisely less depressingly wretched than that of any other system found to date.
(And some of those very smart people get rich by encouraging ad-clicking or exploiting equity mispricings, and then use their money and/or freedom to do neat things from which everyone benefits. Is that such a bad way for things to work?)
Anyway, the "system" I had in mind was the (not necessarily well defined) global one of which the US, the UK and Denmark are all parts.
Speaking as a born USAian, I honestly believe it's more that the USA can and does brain-drain the entire rest of the world. When you actually raise people in an environment of mercenary capitalism, what you mostly get is poor people. When you let social-democratic states across the world raise and educate people, let them fund their research or their early attempts at businesses, and then steal those people when they get successful enough to be an easy bet and put them in a ruthless entrepreneurial environment, then you get a seemingly high concentration of high-talent, high-productivity businesses.
It's kind of like how people outside Israel say we've become a high-tech powerhouse through the discipline of our military training, but actually the Israeli army is one of the world's worst bloated, effort-wasting bureaucracies for every soldier outside a tiny elite.
Except:
1. Fortunately we have taxation to turn highly profitable zero-sum professions into slightly less profitable positive-sum ones.
2. If doing stuff for the quantitative finance industry means more resources for building generally useful Haskell libraries, then everyone wins.
3. If you make a lot of money playing zero-sum detrimental finance games, then you can (a) retire and do more interesting or socially-useful things (like contributing to interesting software projects) and/or (b) give a pile of it away.
On 3b, I'll remark that although I don't work in the finance industry (I've spent most of my career doing R&D for small technology companies), my salary has never been very large by present-day Silicon Valley standards, and my charitable giving isn't especially heroic, I think it very likely that I've done more net good to the world by giving away some of what I earn than I ever have by doing my actual job. If I were to move into finance (which isn't impossible -- I'm a mathematician and quite a lot of mathematicians do that) then it would probably be a net gain in global utility even if everything I did could correctly be called zero-sum detrimental finance games.
Joe is a hedge fund manager. His job consists of arranging for one bunch of random rich people (his fund's clients) to get a little bit richer at the expense of another bunch of random rich people (other people trading in the same markets). For doing this, his clients pay him $10M/year.
The government takes $3M/year of this (note: all numbers in this comment are made-up nonsense), which it uses for building roads and paying schoolteachers and paying for some people's healthcare.
End result: Joe's clients, in exchange for Joe's skill at siphoning money out of the stock market (or wherever) to them, have in effect paid $7M to Joe and paid $3M to build roads, provide medical services to poor people, provide education to everyone, etc.
What controversial assumptions are necessary to make that end result (1) plausible and (2) positive-sum despite the net uselessness of Joe's work?
In some cases one can argue that the useful things done with tax revenue need to be offset against the reduced incentive to work created by the existence of taxes. Not in this case, I think. Joe's work is, to a very good approximation, completely valueless to society, and it does good overall only by being taxed. And if the prospect of being paid only $7M/year makes Joe quit hedgefundery in favour of some easier and less stressful but less well-paid profession, the chances are he'll do more good there. (More precisely: If the world's Joes collectively decide to do a bit less hedge fund management and a bit more engineering or something, that's probably all to the good.)
(It may sound as if I'm criticizing Joe. I'm not. He's not doing any harm to speak of, I think his earnings do a lot of good by being taxed, and of course he may be doing other useful things with some of that money.)
Honestly, the three things I'd most love to have are: - more wrappers, support, documentation on PyMC -- a LOT could be done here and this could replace HUGE amounts of code in all sorts of place - libraries/models on top of cvxopt - more polished ipython notebook -> pdf or d3.js html -- I do this a fair amount for auto eod reports etc
And maybe a fourth...a good open source alternative to kx...preferably one that can accept .h5 files.
I can be reached at hugo@continuum.io if you want to chat about it
I've been developing an interest in statistical modeling and moving those models to production code (and of course, quantitative finance is an "obvious" place where this happens). But I find, even in the statistical genomics world, that the default process still seems to be:
(1) specialized tools for data cleaning and initial crunching (maybe hadoop, maybe HPC clusters
(2) consumption of data from (1) in R or Stata/SAS for model generation
(3) only vague, research-y things where the model from (2) goes to production.
Specifically regarding quantitative finance, it would seem that trumpeting your implementation programming language would mainly be met with blank stares, Jane Street excepted . . .
Take a look at: quantstart.com He has some tutorials with Python to do so, and he is releasing a book (hopefully soon) that should give you a good overview of it all. Also, I recommend you read Options, Futures and Other Derivatives to understand how most of this financial instruments work. There is a lot of good information there.
I'm a big fan of new tools to make mathematical models for traditional investing, but I am fairly against the idea and practice of HFT.
For #1 you need quantitative finance. #2 is the activity known as "market making" and HFT is what you get when you automate it and then let the people doing it compete. The algorithms in HFT are pretty simple-minded; the goal is to offer as thin a spread between buying and selling prices as you can while still making a profit, and to offer it faster (when someone comes looking to trade) than anyone else. The latter is genuinely useful when it's the difference between (say) 5 minutes and 5 seconds. Unfortunately the same incentives that make it worth while for high-frequency traders to offer sub-second response times then make it worth while for them to keep chasing faster and faster times, until the most important asset a HFT shop has is a bunch of computers positioned slightly closer to a big exchange than anyone else's.
There's not much of what people mostly mean by "quantitative finance" in a typical HFT system, I think.
Alpha Heavy Industries is a small(er) SV based firm using Haskell a lot.
Can you give a more specific example? Are you saying that instead of in Java/C++ where you have:
GenericSecurity > GenericOption > EuropeanOption, BarrierOption, AmericanOption
You can have more mutable types/classes at run-time and more injections instead of the concrete ones defined at compiled time?
Also, can you give any concurrency examples in terms of concurrency features? Is it similar to Actor model in Scala? Or based more on concurrent data structures as in java.util.concurrent.*?
Concurrency in Haskell is a big thing. At the lowest level, purity means there's a lot of opportunity for parallelization. Atop that Haskell has a really great green threads system and new GHC has a very, very performant IO manager for running them. It's GC'd so you do have thread-local pauses and they're not as nice as Erlang's actor-local GC. Atop that you've got a nice channel system, the best implementation of Software Transactional Memory around, and neat new libraries like LVish for doing eventual consistency in a very reasonable fashion.
[1] http://research.microsoft.com/en-us/um/people/simonpj/Papers...
I also like to learn concurrency in Haskell and don't know a thing about it (currently, I'm trying to go over Clojure concurrency features). So what you have mentioned regarding the LVish framework and Erlang's actor-local GC, green threading will give me a good starting point to poke around. Thanks again for your note!