Comment by agibsonccc | Hacker News Reader

Comment by agibsonccc | Hacker News Reader

Parent Full thread

It can though. We are a java shop that does HPC and deep learning. Our work is open source. We've been around 2.5 going on 3 years now. Over the years we've surpassed bottleneck after bottleneck in the JVM.

It's a faster wrapper around c code for us. We maintain and created: https://github.com/bytedeco/javacpp

which wraps a lot of c++ components. This is also how we do GPUs and the like with our own memory management for our "numpy for java": https://github.com/deeplearning4j/nd4j/tree/master/nd4j-back...

Short of it "off heap and NO GC" matters a lot.

We have 1 c codebase we use that's meant to be controlled via JNI here: https://github.com/deeplearning4j/libnd4j

The speed gains we've seen are massive. Java can't compete with good ole simd and the like for numerical computing.

We are a big advocate of the JVM as a platform but let's be clear about its weaknesses. You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance.

I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster.

> Java can't compete with good ole simd and the like for numerical computing.

Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too. With 10+ years with C/C++ and 10+ years with Java, I'd bet on Java when it comes to performance bang-for-the-buck almost in every case (given that it's a large app), and even more than that the more concurrent, complex and unpredictable the app is (but this requires carefully looking at the design).

Value types will make Java competitive in absolute terms in more and more domains. Also, with the new JIT (Graal) you can control machine-code generation at whatever level of detail you want.

> You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance.

They're not using unsafe for throughput but (mostly) for latency. That's a whole other matter. Also, some of those use "mechanical sympathy" as a driver of performance instead of algorithms that the JVM makes easier. I've built a concurrent DB in pure Java that relies on synchronization that would require at least double the effort if I were to write it in C (hazard pointers, etc.), and may not even have better performance.

[1]: https://github.com/h2oai

[2]: https://vimeo.com/105743312

>> I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster.

I've had a personal conversation with Cliff himself. Java no matter what you do can't deal with hardware acceleration and gpus. We agreed on that. Numerical software is a different beast. I also kept mentioning "simd instructions" as well as things like openmp.

You are talking about systems software. Unfortunately that matters a lot for machine learning. The axis along which you can get equivalent speeds should be specified here. "values types" != "runs on faster chips"

You aren't likely to beat intel or nvidia's compilers at their own game here . Java will always be playing catch up to last gen's tech there.

Disclosure: I'm more than aware of what's going on in the space. We compete with them for customers and have a very clear understanding of their offerings. H20 has a great k/v store based on the exact mechanics you're talking about. That's about it though. Also of note: Cliff doesn't work on h20 anymore: https://twitter.com/cliff_click/status/700817408110399492

>> Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too....

OpenCL isn't exactly the industry standard for this stuff. You always end up using cuda, and you always end up dropping down to c. There's just no way to avoid that if you want the fastest out there.

Another disclosure, we work closely with nvidia and I may be biased: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-...

I agree with you on the last part, but I keep mentioning "numerical software" for a reason. There are certain things the JVM is good at, writing a database and systems software is one of those things. There are still bits of HDFS in c++ though. I don't think you'll be able to get around having bits of your code in c which is what I emphasize here.

I've also had a personal conversation with Cliff about this (I know he's no longer with H2O), and he isn't (or, at least wasn't when I spoke to him about a year ago) aware of what's going on in Graal. Now, I completely agree that for specialized stuff, it's likely you'll get the most out of specialized compilers, but I'd closely watch the Graal space for specialized code generation. I had a talk with John Rose about how best to combine Graal with the work on VarHandles[1] and Panama[2], precisely to address things like SIMD. There's also interesting work done using Graal to directly emit GPU code (from Java code) for streams, but that's largely experimental at this stage (I think AMD did some work on that and then abandoned it).

[1]: http://openjdk.java.net/jeps/193

[2]: http://openjdk.java.net/projects/panama/

The problem is that you are using it in areas, HPC and deep learning, where using plain C++ would be a better option. I think.

There is a reason why Java 10 aims to improve Java's story in regard to mechanical sympathy.

We wrote a lot of that stack ourselves. And our compute code is in c++. There is a lot of stuff going on in big data land (including us) wrapping a lot of the c++ in java and enabling people to use the JVM for what it's good at (data access). We are pushing this with nvidia: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-...

I also work pretty closely with a lot of the spark/gpu folks at IBM.

While I do largely agree with you, we have our own JNI compiler called javacpp that alleviates a lot of those concerns already: https://github.com/bytedeco/javacpp

Having our own pointer class and doing our own memory management has helped a lot.

What you're talking about is using java for the compute. There's no reason you can't wrap that in a runtime that most people know how to use. A lot of python folks do that now..but then you have to deal with python's limitations at which point the majority of your code (way more than needs be) will end up being in c anyways vs java where you can write a significant part of your app in java and have it be fast out of the box.

Thanks for the clarification.

I am mostly a line-of-business developer doing enterprise consultancy and that is how we use C++, just as infrastructure language when either JVM or .NET stacks need a bit of outside help.

Just wondering if sometimes having a 100% C++ solution would be a better approach than the added integration effort it requires, on the other hand, without people like you guys we wouldn't have access to nice tooling in Java for similar work, so congratulations on the work thus far and all the best for the project.

Reply on news.ycombinator.com