Hacker News Reader

Comment by willvarfar | Hacker News Reader

A very clear and interesting post.

I've been trying to fit big-enough long-running stuff into JVMs for a few years, and have found that minimizing the amount of garbage is paramount. Its a bit like games- or C programming.

Recent JVM features like 8-bit strings and not having a size-limit on the interned pools etc have been really helpful.

But, for my workloads, the big wastes are still things like java.time.Instant and the overhead of temporary strings (which, these days, copy the underlying data. My code worked better when split strings used to just be views).

There are collections for much more memory-efficient (and faster) maps and things, and also efficient (and fast) JSON parsing etc. I have evaluated and benchmarked and adopted a few of these kinds of things.

Now, when I examine heap-dumps and try and work out where more I can save bytes to keep GC at bay, I mostly see fragments of Instant and String, which are heavily used in my code.

If there was only a library that did date manipulation and arithmetic with longs instead of Instant :(

View on HN

> If there was only a library that did date manipulation and arithmetic with longs instead of Instant :(

You can always pass around long timestamps and just convert to Instant whenever you need to do any date/time processing. Provided the Instant doesn't escape the method it's allocated in, it should be optimized via inlining and Scalar Replacement so that it doesn't generate garbage. Of course, you'd be adding in the overhead of dividing up your long in to seconds/nanos each time.

Note: if this doesn't work on OpenJDK, try GraalVM: it's Partial Escape Analysis should do a better job at finding ways of eliding heap allocations.

The worst things happen when value objects are stuck on the heap for a while and then turn into garbage when the value is updated. Escape Analysis doesn't help there, only a good GC can help.

There's a saying that "only the good die young" which applies to Java GC. If your Instants and Strings are really short lived then the GC for those is nearly free. For your workload are these objects living on the heap for long enough to be promoted beyond the young generation?

On G1 you used to be able to use `-XX:+UseStringDeduplication`, which gives you back something like the string sharing used pre-whatever it was (8?).

That appears to still be an option, see the last line of: https://docs.oracle.com/en/java/javase/14/gctuning/garbage-f...

If you are looking to do low latency, then G1 isn't the best choice these days, though. Shenandoah or ZGC are both more advanced algorithms that can greatly reduce the pauses caused by GC activity.

The article compares garbage collectors and with its workloads, Shenandoah had worse throughput, so it depends.

> There are collections for much more memory-efficient (and faster) maps and things, and also efficient (and fast) JSON parsing etc. I have evaluated and benchmarked and adopted a few of these kinds of things.

That sounds very interesting. Can you provide links to the benchmarks for fast JSON parsing (libraries)? And the fast maps?

I can quickly list our findings (from testing on our specific actual workloads; ymmv etc):

For collections, we used to use trove but migrated to fastutil a few years ago.

For JSON parsing, we are processing lots of very small messages, so use LazyJson. The biggest downside to LazyJson is it doesn't have cheap iteration of keys; the framework could easily provide it. For larger documents, say over a few MB, libraries like Jackson are faster.

Yeah, perhaps Java isn't the right tool for our job. And yeah, more recent benchmarking and testing might suggest newer, better libraries than those I have just listed.

Its horrific the lengths you have to go to to get good performance Java for the workloads we have; python prototypes run much faster with pypy, and I think that is really about heap management more than code generation.

For those of us who know C/C++, its kinda uncomfortable when staring at code and thinking "that temporary string there? 40+ bytes just for the object header!" and things. But, of course, there are advantages to working in memory-safe languages.

C# and its structs, yummy.

Thanks!

Wow. I just replaced Java HashMap<Long,Object> with fastutils map for critical cache on a very compute-heavy project of mine, and it instantly got 25% faster. Thank you.

There is another benefit from using the fastutil library, decreased memory usage, which in some cases very significant depending of course on your data structures.

For the last few years, the fastest of the hash map programs shown on the benchmarks game has used fastutil

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> For those of us who know C/C++, its kinda uncomfortable when staring at code and thinking "that temporary string there? 40+ bytes just for the object header!" and things. But, of course, there are advantages to working in memory-safe languages.

And of course, there are memory-safe languages that don't use 40+bytes-long object headers, and are in fact highly competitive with C/C++. For one thing, Go can be made memory safe if you avoid concurrent mutability.

> For one thing, Go can be made memory safe if you avoid concurrent mutability.

golang's performance won't be as good as Java's.

Not if you compare on same hardware but the amount of memory itself that you can end up saving you can re-invest it into compute. I've seen massive perf/$ gains for log-parsing and memory-mapped data profiling use cases.

Not even that is required - the popular synthetic benchmarks have Golang (1.14) beating out Java which is echoed by a lot of users and their usecases. A lot of Java claims seem to come from future/unreleased efforts which muddy a lot of comparisons.

[0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Where is it beating Java by any significant amount? The regex benchmark uses Java's native regex matcher, whereas the golang implementation uses a PCRE regex matcher, so not an apples to apples comparison. The mandelbrot benchmark are minutely different on my machine (2.06s vs 2.13s). Note that the test machines that websites use are quite old (certain optimizations or available APIs the JVM has are not supported by those older processors).

Now if you look at the bottom of the list, you'll see where Java far outperforms golang, especially in the binary-trees benchmark (12.67s vs 9.14s and 25.19s vs 8.28s in favor of Java both times).

Your original assertion:

> golang's performance won't be as good as Java's

My refutation followed with updated synthetic benchmark results.

Your serve I believe. If you have data to prove that, for the workloads in the article, Java would outperform Go I would be very interested in seeing it.

Synthetic benchmarks are not representative of real world performance. People who have dealt with large programs can testify to this. Yet, the binary-trees benchmark is allocation heavy, and typical of workloads similar to those described in the article, and it shows how far ahead Java is able to pull compared to golang.

It's a fact that the JVM's JIT and GCs are way more advanced than what golang has to offer.

> C# and its structs, yummy.

Yep. Java should also be getting value types, hopefully soonish: https://cr.openjdk.java.net/~briangoetz/valhalla/sov/01-back...

C# really hits a weird sweet spot that people on the outside don't really recognize due to C#'s heritage as a Java clone , previous windows dependencies and anti-MS bias of the early 00s.

While i still really can't stand C#'s VB based naming conventions and property happyness the language is extremely practical in many situations.

Writing LINQ you still think mostly in SQL but don't have to worry about string escaping, while on the other side of the spectrum you have structs,spans,stack allocs and in/out/ref vars that gives the code written in the language a chance to keep up with C/C++ on modern CPU's where cache-friendlyness is paramount.

Java on the other hand has opted to avoid much of this and thus has to depend on GC's magically becomming while when working with C# developers can dig down and optimize things on their own instead.

You're absolutely right. Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy.

Everything is a trade-off. Immutable objects enables easier safety when having multiple threads do work concurrently. Shared mutable state is still very difficult to do correctly, and at the point where you're introducing locks then you've crippled performance.

We have so many cores now that it tends to be a positive trade-off to have many threads doing some wasteful work (copies, extra GC pressure, potentially multiple threads duplicating the same work) than trying to have a perfectly optimized single thread.

We have many cores now but memory bandwidth isn't rising as fast. That still makes immutable objects an increasing net loss.

It's just a bad idea.

Disclaimer: I'm not a languages expert, but, I think there's a case to make for performance vs clarity / readability. FP is great for parallelisation, multi-core work, things like web servers and other internet-facing services. But probably not the best for number crunching. Horizontal vs vertical scaling, I think.

I believe you can express your problem (and solution) better using FP, once you have it solved you can zoom in and replace the most demanding segments with iterative programming, or go down lower to the bare metal.

>Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy

It depends how it's implemented. It's possible to get very nice performance with immutability and copying through use of an arena allocator, as your stuff will essentially always be in cache (due to reusing the arena), and allocation/deallocation is just bumping a pointer. Of course, not everything easily fits into this approach, but a surprisingly large amount of code can, if designed with it in mind (and using a language that supports it without too much pain, like C/C++).

The language Zig is particularly interesting in this regard because everything that allocates takes the allocator as a param, and it has built-in arena allocators in the standard lib.

Ideally, a good compiler that understands FP will, behind the scenes, detect when it's safe to mutate the old data rather than creating a copy. That's a big part of why Haskell manages to be neck-and-neck with C despite being functionally pure.

Where it gets tricky is in an environment like the JVM where programming in that style was not anticipated, and introducing any optimizations along these lines for the benefit of the proverbial Scala fans needs to be balanced against the obligation not to adversely impact idiomatic Java code.

That said, even without that, it's not necessarily crazy. It's just a value call: Do you believe that more functional code is easier to maintain, and perhaps value that above raw performance? I'm old enough to remember similar debates about how object-oriented C++ code should be, and to have at least encountered Usenet posts from similar debates about how structured C code should be. I don't bring this up by way of trying to weasel in some "historical inevitability" argument - these are legitimate debates, and there are still problem domains where coding guidelines may discourage, or even prohibit, certain structured programming practices. For very good reasons.

Haskell is only close to C in extremely rare cases or when using unsafe features and the FFI.

Would you say idiomatic Haskell is faster or slower than idiomatic use of Java and the JVM? I'm interested in actual experience and preferably benchmarks of real cases, no thought experiments please :)

(If this sounds harsh, it's not my intention. In another HN thread I had someone "explain" to me how Java and Java's OOP is "not suitable for business software development". If this seems like a bizarre statement which disregards more than a decade of business software development -- this is why I ask for actual experience and not opinions or "I think this can't be right").

“Unsuitable” doesn’t mean it can’t be done. Having written a lot of business software in Java, I actually agree it’s unsuitable. The only way I’ve been able to make it bearable is by using Lombok and pcollections.

> I actually agree it’s unsuitable

How so? Performance? Reliability? Verbosity? Which language is suitable in your opinion?

To me this kind of opinion-based... um, opinions... fly in the face of evidence. Java has been used for more than a decade to deploy business software to great success. What more evidence does one need? It'd be like arguing "COBOL is unsuitable for banking".

In the meantime, opinions have shifted on best engineering practices, and Java has naturally run the gamut of all these opinions. And because there are tons of Java systems, there's no shortage of examples of failed/bad projects one could pick on. I wonder how many languages/platforms would have fared better...

"Unsuitable" meaning it's not a great choice. Yes, it's been done a lot (whether or not you can really say "to great success" is another matter, I think).

What I mean is that I think using Java meant the company was required to spend more resources than should have been necessary to accomplish their goal.

BTW, when I say "Java", I mean the language, not the platform. Java-the-platform is very suitable to business software. Java-the-language, less so. Java OOP brings with it a ridiculous amount of incidental complexity, boilerplate, and impedance mismatches. Of course you can make it work, I do it every day. I just think it's a poor choice. It's certainly not the worst choice, and the platform + available libraries is definitely a plus.

And as I mentioned in my original post, there are way to make it more suitable through various hacks (like Lombok) and libraries, but other languages are more suitable out of the box (including other JVM languages).

If you liked pcollections, you might also like https://www.vavr.io/

These benchmarks seem pretty good. Filterable by language, etc.

https://www.techempower.com/benchmarks/

The source code is published too (https://github.com/TechEmpower/FrameworkBenchmarks)

Lots of people hate Java. It's unsuitable for those people.

There are certain things you might have trouble getting Java to perform, such as strict latency requirements. Another language might be more suitable for that. But that generally doesn't describe business software.

Desktop software is somewhere you probably don't want to use Java. Although if it's business desktop software, it might be a good fit since it can be cross platform (but ugly - but if it's business, it might not matter). We've built several desktop apps for warehouse computers in Java.

Getting the JVM on a machine may or may not be a hurdle. This is one of many reasons why Go is getting popular - you can just build a binary.

The positive of Java is that if you want to do something in it, someone else has probably tried. It has several large organizations backing professional quality libraries and frameworks that have had a ton of resources poured into them. It's easy to build on the shoulders of giants while relying on 3rd party libraries that don't have a bus factor of 1 - this is rare in many other languages.

If you want stability and well trodden paths, it's hard to go wrong with Java. We've had projects that continued to just work from Java 1.4 to Java 8 - a span of almost 15 years without ever having to touch the code. Java 9 was a bit of a hurdle because of project Jigsaw.

> Lots of people hate Java. It's unsuitable for those people.

Understood, but I'm specifically excluding those opinions because they tell me nothing and are unrelated to suitability. Some Smalltalk folk will tell you nothing that is not Smalltalk is suitable for anything; how much would you value their opinion when determining whether a language is suitable for development?

"I hate $LANGUAGE, therefore it's unsuitable for $DOMAIN" is the lowest, less useful form of opinion. It belongs in the realm of flamewars, not of informed decisions.

Suitability to me is not related to whether I hate or like a language. I hate COBOL. I've worked with it. I'd never in a million years argue it's not suitable for banking systems, because that would run contrary to established history.

As for other applications: I agree Java is not suitable for everything. I specifically argued about business software. That said, what about Minecraft, a hugely successful desktop game? :)

> Getting the JVM on a machine may or may not be a hurdle.

jlink[1] makes it pretty easy if you're using a recent JDK.

[1] https://docs.oracle.com/javase/9/tools/jlink.htm

> Getting the JVM on a machine may or may not be a hurdle.

This was/is one of Java's mistakes. It's always been oddly hostile to JVM bundling, although some projects do so anyway (e.g. Jira). More generally, Java makes the mistake of making itself known to the user. The user is expected to install a JVM, rather than one being bundled with the application, and they're then expected to ensure it updates itself appropriately, complete with an annoying taskbar icon and always-resident auto-updater (on Windows, that is).

The user shouldn't even know the word 'Java'. Applications written in Pascal, for instance, are just applications. The user isn't made aware of the technology used.

Especially unfortunate considering that, as far as I can tell, JavaFX is really a pretty good GUI toolkit (I've only dabbled). Perhaps things will change as ahead-of-time compilation for Java becomes more mainstream.

> Perhaps things will change as ahead-of-time compilation for Java becomes more mainstream

I understand JIT compilation is pretty advanced these days. Wouldn't this go against it? Or maybe the approach can be mixed, but if so, you'd still need the runtime environment ("the JVM").

If you ever try to write a bash script that calls a Java program in a tight loop, you'll know the limitations of a fat interpreting runtime with JIT compilation. Another relevant use case is the Stateless Lambda server-side architecture.

I believe Java (OpenJDK at least) is slow to start even if you disable the JIT and go with pure interpretation, and even if you disable runtime bytecode verification. It's just generally heavyweight and slow to get off the ground.

I imagine JIT compilation should help with this, as you suggest.

Oops, too late to edit, but I meant ahead-of-time compilation should help with this

A mixed approach can be done, yes. If dynamic classloading is needed, you need to bundle a JIT as well (or at least a traditional interpreter). Excelsior JET has used this hybrid approach for years. [0] I imagine it should be possible to omit the JIT if it can be determined that it's not needed.

[0] https://en.wikipedia.org/wiki/Excelsior_JET

They (Oracle and Java) are going the other direction on this. Jigsaw helped with JDK bundling. IntelliJ bundles its own distribution.

Excellent, I see I'm behind the times.

Since Java 8, there are two ways to build a binary: the javapackager tool, and the Ant JavaFX tasks (with the OpenJDK, install the openjfx package). It'll include the JRE [~50MB], and there are pros and cons with that obviously.

Which person's idea of idiomatic ?

"One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of reasonably idiomatic Haskell. However, we have made abundant use of strictness annotations, explicit strictness, and unboxed vectors. We have, more controversially perhaps, used unsafe array subscripting in places. Are our choices reasonable?"

http://www.leafpetersen.com/leaf/publications/ifl2013/haskel...

Programs for the n-body and spectral-norm procedural benchmarks game tasks, perform kind-of the same in Haskell and Java

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> That's a big part of why Haskell manages to be neck-and-neck with C despite being functionally pure.

Come on... that's not what you typically get with Haskell, is it.

Rust gives you FP-style without loads of copies and C-like performance. Immutable data-structures aren't idiomatic, but it's ownership gives you most of the same benefits.

And many string types, like Haskell /s

The nice thing about Rust is that it gives you the main benefits of FP even when you're not programming in "FP style". The Rust "sharing xor mutability" default model provides underlying semantics and ease of analysis that's quite comparable to what you get with a pure-functional language. Of course extended mutability as with Cell<>, RefCell<> etc. undermines this, but these are only used when necessary.

> Ideally, a good compiler that understands FP will, behind the scenes, detect when it's safe to mutate the old data rather than creating a copy. That's a big part of why Haskell manages to be neck-and-neck with C despite being functionally pure.

Not talking against Haskell or FP; The disadvantage is being hard to reason about performance. Ensuring compiler will optimize something will take more cognitive load than simply writing straightforward code.

Imagine having a map() method on an array that produces another array. In a long chained pipeline of such things, compiler may be able to elide extra allocations and generate asm very near to handwritten for loops. But the abstraction breaks when the method does I/O or has side effects - you can't reorder anything in order to elide allocations. (Well this example may not make sense in Haskell because Haskell is lazily evaluated).

However there is a limit to what compiler can do, and it might manifest in edge cases like variation in order of imports. I'd rather have straightforward code than relying on compiler optimizations.

I am not arguing "against abstractions" like some Go fanboys tell you map and filter are less efficient. It is always possible to apply same map filter to iterators/lazy streams instead of arrays and get same performance in straightforward code. But that's not same as keeping in mind what heuristics compiler uses to optimize the code.

I suppose in FP/Haskell you would structure your code differently, so that it wouldn't have side effects when mapping arrays. That way you could even parallelize it trivially.

High level languages produce a lot less code than than less optimized implementations. Difficulties with reasoning about execution/performance is the penalty you potentially have to pay.

In my experience it indeed may be hard to understand performance with Haskell. That said, it comes with excellent tooling to overcome it in most of the cases.

If an object is truly immutable, its object identity shouldn't matter in the vast majority of cases, and so it should be okay to just copy the whole thing, instead of passing around references to it.

Unfortunately, the legacy Java semantics of == means that they can't do this proactively. But didn't Java get opt-in value types recently?

Persistent data structures give you a mixture of performance and immutability and are common in functional programming.

Reply on news.ycombinator.com