Hacker News Reader

Comment by shmerl | Hacker News Reader

Why is it written in Java? Isn't it a default performance hit? I'd expect such frameworks to be written in high performance languages.

View on HN

Java is pretty high performance, usually on par with C++, and usually faster than e.g. Go.

Writing Java code so that there are no perceivable GC pauses is an art, but it is not impossible to achieve.

JVM might require more RAM upfront, but a well-written program is usually reasonably memory-efficient, too, so the consumed memory grows reasonably slowly with the problem size.

Writing things in pure C is often just too time-consuming.

> Java is pretty high performance, usually on par with C++

I'm not convinced. Java I/O is far form perfect, and Kafka is probably very heavy on I/O side.

> and usually faster than e.g. Go.

That's strange, since Go to some degree was intended as replacement for Java without having Java's downsides. Why would Go be less performant?

I'd be interested if someone would write such framework in Rust though. C++ is of course a default expectation, but usage of Java somehow surprises me in this case.

> Java I/O is far form perfect

The API certainly isn't perfect, but what do you find lacking about the performance of Java I/O?

http://docs.oracle.com/javase/7/docs/api/java/nio/channels/p...

Why are you so deadset on assuming that Java is slow? That's an incredibly misinformed and narrow view of what's possible on the JVM.

I didn't say it's just "slow". I said it's an expected performance hit in comparison with languages like C++. In some cases that hit is tolerable, in others not.

Right, but even what you said isn't really within shouting distance of reality.

Certainly, many naive people would expect such a performance hit; personally, I'd strongly expect that the actual performance is predicated on the architecture and the expertise of the teams involved and that any differences between the languages become either apples to oranges or noise.

I'd expect an actual performance comparison to be sure, rather than relying on expertise of the teams for assuming it. As much as any claim that difference between languages becomes irrelevant with advancement of the hardware, it still can be relevant in different cases.

Kahka is written in Scala, not Java. The only Java in the project is for a JavaAPI and hadoop connectors.

Comparing languages in absolute performance terms is bad idea, it's an extreme simplification of what really goes into creating performant applications.

> Kahka is written in Scala, not Java.

Thanks for the correction.

Definitely, definitely not true. Go is extremely fast, it has its strengths in some areas and trade offs in other areas. If you want a good comparison to Apache Kafka, you should look at Apcera's NATS server, a publish-subscribe and distributed queueing messaging system written in Go by Derek Collison: https://github.com/apcera/gnatsd

You should never discredit a language, especially with blanket terms such as "its faster than Go". In what respects and in what areas? Here's a blog post which performs benchmarks on Go and Scala: http://eng.42go.com/scala-vs-go-tcp-benchmark/

They found Go to perform better than scala, however it had a high footprint. Every language has its tradeoffs, Java and Go are no exception.

Your link says the opposite of what you posted. From the article: "To our surprise, Scala did quite a bit better than Go, with an average round trip time of ~1.6 microseconds (0.0016 milliseconds) vs. Go’s ~11 microseconds (0.011 milliseconds).", and "Notable in the opposite direction was that the Go server had a memory footprint of only about 10 MB vs. Scala’s nearly 200 MB."

I have no idea where people got the idea that Go was faster than Java, or that Java/JVM is slow in 2013. Not trying to discredit Go (its my language of choice), but to say it surpassed Java while only being around for almost 5 years is disingenuous.

nemothekid,

You're absolutely right, my apologizes, its been a while since i read that article. Thank you for the correction! Much appreciated.

Java I/O is plenty fast. Just take a look at Hadoop, Lucene, Jetty, Netty, or any number of other pieces of high performance software written in Java.

Here's a good writeup talking about sequential I/O in Java: http://mechanical-sympathy.blogspot.com/2011/12/java-sequent...

Thanks for the pointer. Is there any comparison with C/C++ I/O in various operating systems?

Note, that unlike Hadoop, original Google's map reduce system was written in C++.

I/O is I/O: the big differences are lack of certain APIs (which could make it poorly suited to some -- but not majority -- of I/O intensive use cases; needing -- afaik this isn't resolved with NIO.2 -- needing to use native bindings for kernel AIO is one example), poor APIs (ByteBuffer, I'm looking at you, but here's an alternative: https://github.com/airlift/slice), and the fact that by IO (obviously) involves making JNI calls which do have a latency penalty, but one that is dominated by costs of I/O itself (if code is well written, we're speaking nanoseconds here -- while even a FusionIO device would have latency measured in microseconds). I would say for bulk throughput oriented transfer from disk to socket via sendfile(), I'd say the costs of using Java are not noticeable at all.

Re: Google. MapReduce paper was published in 2004, only a few years after Java 1.4 was released. The infrastructure it is built on top of -- and like M/R itself -- were most certainly written before Java 1.4 was released and likely at a time where running Java on Linux meant using Blackdown -- which had its own issues. Java 1.4 is when java.util.concurrent and non-blocking I/O were introduced; prior to this, writing scalable socket in Java was far more difficult. It would also be until Java 1.6 that epoll() would be supported by Java on Linux, etc...

The other big part is that Map/Reduce is not in a vacuum: while Map/Reduce workloads are mostly I/O dominated, other pieces of infrastructure built on those working blocks are very memory intensive. Google "Java GC" to why that is still an issue with Java (despite having an extremely advanced concurrent garbage collector) In addition, I'd machine in the case of Google the performance advantages of C++ (more about things like being able to lay memory out in precise ways and being friendly to the CPU caches, rather than about pure performance) really do matter -- as Joshua Bloch put it "inner loops of indexers would likely stay in C++".

Finally Google does have at least some infrastructural pieces that in Java: I do know that there are first-class supported APIs for BigTable/Spanner and Map/Reduce in Java, there is also the FlumeJava paper (an EDSL for Map/Reduce in Java), and while I believe C++ infrastructure has superseded MegaStore it is one piece of Java being used for Google's core infrastructure.

That's not to even mention Google's well known exteandnsive use of Java for web-applications/middle-ware including AdSense/AdWords billing and management and gmail.

In the end, you can certainly write I/O intensive applications with Java, but with some caveats: be sure it's actually I/O intensive and the non-I/O intensive parts perform fine in Java, be wary of GC hell, know how to avoid excess/implicit copying (pretty much a case in any languages), and be sure that the APIs you plan to use are available.

In the end, I don't think it's an either/or answer: given (hate to use a buzzword, but it certainly applies) service oriented architecture, you can implement various pieces of a system in languages best suited for that task. This is becoming easier in the H* world now that protobufs is the "native" RPC language as opposed to Java serialization.

Reply on news.ycombinator.com