> antiquated VM
I'm always flabbergasted about the ignorance against the JVM on this site. There is nothing out there that has seen similiar research and optimisations and has so good monitoring/debugging tools.
I'm always flabbergasted about the ignorance against the JVM on this site. There is nothing out there that has seen similiar research and optimisations and has so good monitoring/debugging tools.
It shouldn't have happened in a production environment, but setting up a way to lose tens of thousands of dollars if performance is bad, trusting unreasonable numbers, trying to produce no garbage, and requiring no GC pauses because margins are too tight are all your faults, not a shortcoming of your JVM.
It's my fault that if you trade US equities based on old information you lose money?
Edit: Actually, I'm still not sure how to correctly measure GC pauses, other than from inside the application.
Can you explain how Java was chosen for your trading application? Also, why were you unable to test it properly?
The application was tested in an environment that's a lot like production, except that we had substantially fewer cores than an exchange running the same number of matching engines, so the incoming ticks were a bit less bursty. That burstiness was important in this instance. A big reason for the differences between the production environment and the test environment was the cost of hardware.
No criticism, as I've only read others' accounts, but that sounds out of character.
What's "out of character" (nice euphemism!) is adopting state of the art automatic garbage collection technology, writing in a language that makes actual garbage collection practically unavoidable, and then expecting that garbage collection doesn't happen or happens accordingly to arbitrary expectations. The JVM is clearly inappropriate technology if latency is important: it can perform well in common cases and with a reasonable level of tuning effort, but other options simply do not have the threat of GC pauses.
I'm sorry to say it, but the loss of money here was not the JVMs fault. It was putting code with insufficient load testing in a production environment and crossing your fingers despite the fact that serious money was on the line.
Considering that software mishaps in securities trading can annihilate a company of any size, not only cost you as much as a bunch of servers, lack of testing appears very reckless.
We did not generally physically swap out servers each time we changed a line of code. Rather, we had two identical sets of servers, one for production and one for test. For test, we had some additional machines to run matching engines. If we wanted to make the test environment more closely match an actual venue, we would need many more machines that run matching engines. That still might not be sufficient to produce identical output timings in prod and test, though, because venues generally do not publish the sequence of all incoming ticks they receive during a day, the source code of their matching engine, what hardware they are using, their kernel version, and so on.
> Considering that software mishaps in securities trading can annihilate a company of any size, not only cost you as much as a bunch of servers, lack of testing appears very reckless.
It's strange that you replied to me suggesting we had no production-like test environment ~10 hours after I replied to you saying that this code did fine in our production-like test environment.
Can you jump over 64Kb in bytecode yet? Allocate a lexically scoped variable?
Of course, enormous amount of work has been put into making this particular pig fly at reasonable speed. Doesn't change the fact that most usability complaints about Clojure stem from reliance on JVM piggybacking.
Tuning can be a pain, but scaling anything is hard.
Kotlin, from the little I've seen, appears to be largely a syntactic variation of Java, so there is certainly good impedance to JVM. Clojure is less so, and here lies the reason everyone curses about its backtraces.
Clojure is in a bad spot here, it was a good idea at the time but not today. Clojure relies on Java for its libraries cause Clojure has almost no ecosystem. 95% of Clojure libs are Java wrappers.