Caml Trading – Experiences with OCaml on Wall Street [video] | Hacker News Reader

Caml Trading – Experiences with OCaml on Wall Street [video]

infoq.com

93 pointsgnuvince42 comments

Quick question for the ocaml crowd: what keeps you with ocaml when you have a very similar and more easily parallelized f# available? I ask as an outsider to both, but a fan of ML-ish languages in general. Would Jane Street ever be open to using F# or Haskell or Rust or Scala?

I'm only a recent dabbler in OCaml, so take my opinion with a grain of salt. First off, it depends on what kind of problem you are working on. Support for concurrency is good (through Lwt and Async), so the situation is not worse (actually, better, because monadic concurrency is vastly superior to callbacks) than in a popular language like Javascript, or Python with its GIL. Secondly, work started on a parallel runtime a few months ago, and seems to progress at a steady pace, so we should get parallelism and cheap fibers at some point.

Regarding other functional languages, you are mixing some oranges in your apples. I find Rust appealing, especially with the amount of thought that goes into crafting it, but when it comes down to it, it is intended to replace C++. You are trading the convenience of a garbage-collected runtime for more control over memory allocation, and this comes at a cost. I could imagine OCaml calling Rust code for performance-critical sections, though. Regarding F#, it's no doubt a nice language and works as an incubator for ideas which trickle down to C# eventually, but I for one have little interest in the .Net ecosystem. Scala I find too scattered in terms of features, it doesn't make for a very elegant language. Also, the JVM is performant, but it's a pig in terms of memory. As for Haskell, many OCaml developers have been moving to it in the last few years. On the other hand, you don't switch to language with a lazy runtime with impunity (debugging Haskell is not particularly fun). I find the average complexity of Haskell programs higher, and a combination of syntactic features, popularity of custom ASCII operators and one-letter variable names make for sometimes barely readable code. Not to mention that OCaml is more pragmatic: sometimes it is convenient to write a for loop.

Rust actually has an optional GC. Also, even non-GC code in Rust is a lot different than say in C++ since the compiler can point out a lot of memory leaks to the programmer.

I'm well aware of that. However, using GC fattens considerably Rust type signatures (want a garbage-collected list of garbage-collected Foos? Enjoy writing Gc<List<Gc<Foo>> everywhere). It's just not particularly idiomatic. And Gc types are not Send types, so you can't share them across tasks, which is unfortunate since we were precisely discussing the lack of parallelism in OCaml :)

It's not a dig at Rust, which I think is very well designed, but IMHO, it is in a different niche compared to the other functional languages listed above.

"...writing Gc<List<Gc<Foo>> everywhere..."

Rust, unlike Java, has type definitions[1]; there's no need to use the full expansion everywhere.

(I don't believe it's even necessary in Java, but creating subclasses has its own downsides.)

[1] http://doc.rust-lang.org/0.11.0/rust.html#type-definitions

Yes, you can do that, but it's pretty obvious that if you want garbage collection everywhere, you're not going to use a language with linear types and optional, pretty much discourage GC, and write type synonyms all over the place (or fat signatures), you're going to go for a language where it's natural.

> Rust actually has an optional GC.

Last time I checked (0.10 or so) it did not. Gc pointers were the same as Rc pointers with a proper GC to come soon.

Rust has roughly the same problem as C/C++: without compiler support for garbage collection, you quickly get off into the unappealing-option weeds. In this case, either "conservative" collection, or smart-pointer based collectors, which have performance issues compared to "proper GC's". (As far as GC features go, you can get pretty far with smart pointers: http://maniagnosis.crsr.net/2014/03/automatic-memory-managem... .)

At 48:10, the presenter briefly talks about the plausibility of using F#, Haskell, and Scala at Jane Street.

Very interesting, in terms of pragmatism OCaml was the best choice for them at the time but now Minsky would also consider using Haskell or Scala. I guess Scala might arguably be the most pragmatic of the lot (access to tons of high-quality Java libraries, etc)?

OCaml is not my primary programming language, but I have been looking at it among many other options including all of the ones you mentioned.

The module system of OCaml is very powerful, and a similar module system is not present in F#, or anything else besides some variants of StandardML as far as I know. I was able to accomplish many things using Functors that I have not been able to accomplish in any other languages.

Here's a link to one of my previous comments regarding some of OCaml's compelling or unique features: https://news.ycombinator.com/item?id=7861033

At the time of writing that, there were a couple of reasons why you still might consider Haskell instead of OCaml (assuming you were willing to accept complete purity and were willing to adapt to lazy evaluation). Those two primary reasons were type classes and a multi-core runtime (where immutability would really shine). I don't know much about Haskell, but this is what I've heard from other well informed people. I'm sure there's many other great reasons to use Haskell over OCaml, but these were the main ones from what I could tell.

But since then, it looks like there's been great progress on an OCaml multi-core runtime, and also something very similar to type classes (module implicits). We'll have to see how those develop but it's definitely exciting.

Also since then, Jeffrey Scofield has accomplished a lot to get OCaml working on modern iOS systems so you can build native iOS apps with OCaml: http://psellos.com/ocaml/compile-to-iphone.html

I've been building a small project in OCaml for a while now, and my experience hasn't left me wanting any core language feature that some other language has, aside from possibly a multi-core runtime (I haven't actually needed it yet, but I will soon). Most of the things I wish were different are very superficial syntactic gripes - but OCaml supports arbitrary syntax via ppx extensions so I can just stop complaining and build the syntax I want to program in. (Haskell-inspired syntax? Swift-inspired syntax?).

The only other feature I'd like to see is a unification of the object system and record system: You could describe that as "row polymorphism" built into records (complete with pattern matching). No other major contenders support this, so it doesn't effect my language decision, yet it would be a welcomed feature.

> The module system of OCaml is very powerful, and a similar module system is not present in F#, or anything else besides some variants of StandardML as far as I know.

Scala's module system is very similar to ML's.

    Object        ≅ Structure
    Class         ≅ Functor
    Trait         ≅ Signature
    Abstract Type ≅ Abstract Type
    Refinement    ≅ Sharing Constraint

This is roughly correct, but see Andreas Rossberg's caveats here[1] for some considerations that may be important depending on what you're trying to do.

[1] http://stackoverflow.com/a/23019436/82959

Great point. And many of these distinguishing features mentioned in that post are the ones that I used in order to accomplish some very powerful abstractions that I couldn't have done in any other language that I've tried. (Functors operating on types and values).

Excellent overview, OCaml is quickly becoming a powerful and flexible tool in almost all aspects comparable to other languages and has its own unique strength like module functors and polymorphic variants.

> The only other feature I'd like to see is a unification of the object system and record system: (...)

Could you elaborate a little bit on this or maybe point an article about it?

Thanks.

I'm not the best person to elaborate, but I'll explain from my limited knowledge.

OCaml actually has a very complete object system (with ugly syntax, but don't judge). It's similar to OO you're used to in Java etc, but the types are often inferred instead of requiring that you state up front that classes/objects are subtypes of other classes.

It allows you to use objects just as you would records, except:

1. You don't have to predefine the object types anywhere - with records you do. 2. With OCaml "objects", you get inferred "row polymorphism", which means that you can make a function (without any type annotations) that accepts a point, and accesses point.x, and point.y. This function will also, gladly accept a 3D point that has an extra field point.z, that the function never accesses. In OCaml, you never even needed to predefine a type for 2DPoint or 3DPoint. This is similar to, but different than the OO "subtyping" that you're used to in OO languages. 3. Records don't provide this "row polymorphism" that OCaml objects provide. 4. Records do allow pattern matching. 5. I would like to see the best of both. Pattern matching and row polymorphism into one new record abstraction.

I'm sure there are tradeoffs (likely something to do with predictable memory layout accommodating efficient instructions). Row polymorphism may make type inference times longer, or error messages less readable. I'm definitely not the right person to give an overview of this - so please do your own investigation on the tradeoffs. I'd love to hear an expert chime in.

But all in all, I'm happy with records and the performance I see. Named arguments (with optional defaults) (which OCaml has) can fulfill the role of a unified record/object system, in some cases.

> But since then, it looks like there's been great progress on an OCaml multi-core runtime, and also something very similar to type classes (module implicits). We'll have to see how those develop but it's definitely exciting.

Is there any information about these that I could read?

Modular implicits: http://www.lpw25.net/ml2014.pdf

The multicore runtime: http://www.lpw25.net/ocaml2014.pdf

Recent presentation on the multicore runtime: https://www.youtube.com/watch?v=FzmQTC_X5R4

Some things I'd like to see are a single unified stdlib, more use of sequence types and have the toolchain streamlined (death to Makefiles).

Haskell is getting row polymorphism[1] for records (going under the name OverloadedRecordFields) in the next release. The new language extension will also play well with lenses, making records even more versatile.

However, we're still not getting special syntax for field access, so it'll be function application (g f) instead of a dot (f.g). This isn't a major drawback, but it means that a field g has to be in scope for this to work at all.

Fundamentally, it seems exactly the same approach could be used to give Haskell polymorphic variants. They're not part of this extension, but I wouldn't be surprised to see them in the near future.

Haskell is also getting a much better (ie not crippled) module system soon, in the form of Backpack[2]. However, I'm not sure when this is actually going to be released—I don't think it's in the upcoming version.

[1]: http://www.well-typed.com/blog/84/

[2]: http://plv.mpi-sws.org/backpack/

From the talks I've seen Gundry usually tends to qualify comments about OverloadedRecordFields by saying it's not traditional row polymorphism ( i.e. Record { name :: String | r }) ). It doesn't allow row extension or row restriction as first-class constructs in the type system. Although we can build these things as secondary features on top of the language as many libraries currently do though.

When you say it's not first class, you mean that it can only be inferred?

Inference is orthogonal, I mean that a records don't have a distinct type in the type system. So structurally the following types are the same in Haskell, except one elaborates out a selector function.

    data Foo = Foo { name :: String }
    data Foo = Foo String

What row polymorphism does is give use first-class record types that give us a measure of structural typing so that records with the "same" fields can be used in the same type and "larger" records can inhabit "smaller" types if the fields subsume. So we can write down a function like:

    foo :: forall r. { a :: Int, b :: Int | r } -> Int
    foo x = x.a + x.b

The current proposal doesn't give us this. But we can still simulate it even if it isn't in the language.

F# runs on CLR. I use a Mac.

(Yes, I know about Mono, but I'd rather use something officially supported).

Also, I don't see a real difference between CLR and JVM, and I won't support another VM that purposely breaks up the ecosystem.

(On the other hand, if MS continues the current trend and goes completely open source/free software (edit: and adds an ahead-of-time compiler), I'll probably change my mind.)

I believe F# runs on Mono, thus mac and linux.

F# can only create .Net code AFAIK, and according to wikipedia it supported Linux only since 2010. I have already decided to learn/experiment with OCaml by that time, and I haven't payed much attention to languages that can only run on .Net/Mono because Linux will probably always be a second-class platform for them.

Somewhat haphazardly:

* You don't have the C# baggage

* You have a modern module system, which is only paralleled in Standard ML. This is very important for large developments where the need to split your code base into manageble chunks is extremely important.

* The sequential code generator is very good.

* Compilation times are fast. Not Go-fast, but reasonably fast. Building OCaml itself with OCaml can be done in around 1 minute on a fast machine.

* The language is strict! This makes some gotchas w.r.t memory behaviour much easier to reason about, and is a definining distinction towards Haskell

* The language is rather small in the core. In contrast to Scala.

* The Garbage Collector is state of the art. It is fast. It has small pause times. It doesn't overuse memory.

* The language is mature, which Rust isnt'. When Janes Street started, Rust did not even exist.

Very cool. I was very close to going all-in with OCaml a few years back when I made the switch to FP. I still love their module system -- and am very disappointed to see F# leaving its roots and becoming just another .NET language (which has it's good parts and bad parts)

What swayed me at the time was the fact that you had to roll your own libraries for so many things that you get out of the box with .NET. I understand that's changing, but in my mind it's just not ready yet. Maybe in a year or two. I'm still thinking of moving on to something else. It's either going to be OCaml or Haskell.

Would love to see some folks doing this same kind of thing with F#, talking about real-world production systems.

I've done some small app work with it, and blogged about it ( http://tiny-giant-books.com/blog/real-world-f-programming-pa... ), but what the F# community could use is some fans like OCaml has with Jane Street.

OPAM, the new package manager, is very good. But, yeah, the ecosystem isn't that huge.

Out of interest, what were some of the major things you found missing from the OCaml library ecosystem?

Remember, it was 4-5 years ago, and OPAM didn't exist.

I remember thinking, "Wouldn't it be neat to scrape some web sites?" I couldn't find anything in OCaml, but .NET had the HtmlAgilityPack written in C# for Windows. Pop over to github, pull down the source, recompile? Badda bing, badda boom, it's running in Linux. And I'm able to use it in F# without really having to know C# (Even though that's not the case). I realized that I could do that for literally thousands of different apps and utilities. So you had this entire ecosystem of programmers making open source stuff for windows that you could easily use in linux. That was the deal-closer.

Plus it looked like Mono had finished going through it's almost-ready-but-not-quite stage. I would still love to be able to permanently load .NET apps in Apache and call them a la fastCGI, but heck, even that might be a solved problem by now.

Finally there was the F# community, which was basically a bunch of old OO MS hacks like myself trying to find their way along mixed in with some really nice academic types who were helpful. OCaml, sadly, did not have that kind of feel. Instead there was a lot of French. And searching around the web for answers to how to do really simple things, like get started, write a web app, or cross-system develop.

The barrier to entry was just too high.

But, like I said, that's changing.

In what way has F# become just another .Net language?

They really should have adopted functors and the OCaml module system. By not doing so, it's leaving too much coolness on the table. Instead, they did this thing where every file is actually a module, but not really. Kind of an OO mess. If they are not careful, they're going to end up in Scala hell, where it's FP sitting on top of OOP, instead of a true hybrid language. (Yes, I know it's all CLR under the hood, I'm talking about how it feels to the average developer)

> If they are not careful, they're going to end up in Scala hell, where it's FP sitting on top of OOP, instead of a true hybrid language.

Maybe there are better hybrid languages, but compared to Scala I wouldn't even try calling OCaml or F# hybrids. OO/FP interop in those languages feels like the language designers were actively sabotaging it.

He also persuaded the traders to use emacs.

I couldn't get the video or presentation to play, even in Chrome with Flash enabled, without using a mobile user agent. And you can't download the audio or slides without an account. Great web design.

http://d1snlc0orfrhj.cloudfront.net/presentations/14-jun-cam...

Works for me. Have you tried in p0rn mode?

My girlfriend found a jane street promo t-shirt in an Ithaca thrift shop - it has a nice starry sky pattern and reads something like "A more elegant language for a more civilized age". I'm enamored with the shirt! I don't know much about performant FP financial wizardry but appreciate their design dept.

https://www.janestreet.com/blog/what-we-do/faqs/ <-404's for me

Wow I would love a Jane Street shirt

I know, right? It's one of those companies that depresses me because I think I'd love working there but I don't consider myself a genius

I don't believe you need to be a genius. Just willing to move to New York. Give it a whack. How much can it hurt?

I got given one at ICFP this year. If you're in the UK I'll send it to you.

Reply on news.ycombinator.com