Hacker News Reader

For .NET you should really read about structs, which allocates on the stack instead of the heap, and thus avoids the GC most of the time. Oracle is working on a struct implementation for Java 9 or Java 10.

Using an arena/buffer/cache in Java and .NET is not unheard of, and is a good way of avoiding allocations, and thus postponing collections. An arena will, however, increase your live memory which means that when a collection finally does happen, it will take longer. The arena will probably be promoted to the oldgen quickly though, so the collector shouldn't scan it to often.

"if nothing points into this entire arena dump it in one chunk" -> is more or less how a GC works, so you get that for free. The way the GC does it, is that it scans everything that is live and considers everything not scanned as garbage, which is then collected. The beautiful part of this, is that even if you have allocated 1Gb of memory, you only pay for what is live. If you have 200mb of live memory, collection will be constant regardless of how much garbage you have, whether it is 1Gb or 2Gb of garbage. Increasing the size of memory of course makes collections happens less often.

Your point regarding JIT is irrelevant in this particular case, as this case is exclusively related to the GC. We would be talking about the same potential speed improvements here with or without a JIT. That being said, you are also wrong.

A JIT can produce faster code than any pre-compiler, due to there being more information available, and because a JIT can easily "undo" compiled code, allowing it to experiment with code that can fail in certain rare cases, but most of the time is vastly faster. Mono, the open source .NET implementation, can be pre-compiled using LLVM (originally a C/C++ compiler), but using the JIT produces faster code.

The reason Java and .NET is "slow" for idiomatic (not higly optimized, which most code isn't) code, is due to the GC, which causes the application to freeze at certain intervals. This problem increases the larger live memory you have. A GC is probably faster overall compared to manual memory management though, the problem with GCs today is that they de-allocate (garbage collect) all at once, instead of de-allocating memory once said memory becomes garbage, which evens things out a bit.

That being said, there are of course downsides to a JIT. It takes time before the JIT is "warm", which is to say it takes time before most of the code in the application is compiled. If you have applications that run for a short amount of time (I think Java's JIT compiles functions that are run 32 times or more), like shell scripts, they get no performance benefit of using a JIT. Another thing is memory use. Since a JIT compiles code as it goes, and nothing is pre-compiled, the JIT has to store compiled code in memory. So a JIT will naturally use more memory than a pre-compiled program, and a GC normally ads to this.

Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects. So your GC run ends up not being great anyways. Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns. But if there was a simple way to stack alloc objects when safe, that could mean a lot, and is certainly faster, and is hassle free- barely worth calling manual management. In fact, there is a way to do this, by using unsafe code and setting up your own object headers; except the GC won't scan your object, so you gotta be extra careful.

The line about the JIT emitting better code is often repeated. I used to repeat it myself. But diving into the emitted code, I'm not overly impressed. So, sure, in theory, JIT codes going to be all amazing, taking advantage of my cache sizes, common flows, etc. In real life, this doesn't happen. As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place. Structs even used to prevent optimizations in the CLR codegen!

"Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects."

Of course you don't have control whether to place an object in the nursery or the oldgen, that is an implementation detail, but that has nothing to do with the nursery being shared across all threads. Erlang is, as far as I know, the only VM that doesn't share GC space between threads (actually, processes) but has to rely on string-copying, which has it's own issues. Even here, you can't decide where to place an object.

"So your GC run ends up not being great anyways."

When I worked on a C/C++ project a couple of years ago, me and a colleague spent six weeks tracking down a memory leak. I've considered GC runs to be the most amazing thing in software ever since.

"Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns."

The work a GC does is strictly necessary, and the collection time isn't really bad (depends on implementation of course), the only places you might have a problem is in realtime apps like games. In this case however, the GC related problem seems to be quite easily remedied by rewriting some code, and thousands are using a several year old, and crappy, GC in unity and mono 2.x. For most applications these days, the GC really isn't the issue, the code is.

"But if there was a simple way to stack alloc objects when safe"

Value types, Coming Soon to Java, already available in .NET. Of course, using these in the wrong places can prevent or work against later optimizations. Go gives you greater control over this, but that is Go.

"But diving into the emitted code, I'm not overly impressed."

There are certain things to consider here, like tiered compilation. Another thing to point out is also that the exact code you were looking at was difficult for the VM to optimise, or that it didn't run so often that the VM found it necessary to optimise as much as other functions.

We had some developer tool we ported over from C++ to Java, more or less a straight copy of code, with changes to match the different APIs, the Java version was consistently faster (given enough runtime). Sure, you could probably make the C++ version run faster given enough skill and time, but then it wouldn't be idiomatic code anymore, and there the Java version excelled.

"As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place."

There are several reasons why this might be the case. HotSpot has (I believe) an 'inline-budget' which means it only inlines functions less than a certain bytecode size. Similarly, there are other limitations on what code the JVM will optimise. Just because manual inlining worked in your particular example, does not mean you will get the same benefit every time.

"Structs even used to prevent optimizations in the CLR codegen!"

Who cares about 'used to'? It's normal that quirks are worked out over time, and that things improve. By the same logic, it's normal that Java produces faster code than .NET, and it's normal that GCC/LLVM produces faster native code today than ten years ago.

That's actually not true for .net. The tiny local variables can be structs rather than objects. For example if structs could be used for coordinates in this change instead of full objects, they would likely never touch the heap.

Except it is true because structs are often frowned on, and plenty of things end up as reference types. And arrays are reference types too, so things like string.split have no workaround. Also note that until v4 or so, the CLR had limitations optimizing methods using structs.

But yes, the CLR has a far superior design and can avoid some of Java's pitfalls.

>structs are often frowned on

Only in the premature optimization sense, and "familiarity with new programmers" sense. There's nothing wrong with using structs. In fact, XNA and many other game programming libraries do use structs to represent position vectors, which is the context of this article.

>And arrays are reference types too, so things like string.split have no workaround.

If you're okay with going unsafe, arrays can be stackalloc'd.

Anyone writing Minecraft in C# would use a struct for the position vector. There's no good reason to frown upon it in this case.

It's not C#, but here's Minecraft in C++ and Lua: https://github.com/minetest/minetest

This is a cultural problem. We use structs all over the place in Roslyn. You need to train your developers on how to appropriately use the features available to them on the platform.

Structs are frowned upon? Since when?

Hopefully Java will eventually get "value types" too: http://cr.openjdk.java.net/~jrose/values/values-0.html

Hopefully they make a way for arrays to be stack-allocated and passed by value too and take a step ahead of C# in this department. :)

In C#, you can kind of do this with stackalloc and fixed arrays, but it is still not as ideal as could be.

Tangentially related, but there's a Minecraft clone being written in Rust. Still very early stages, but pretty cool.

https://github.com/PistonDevelopers/hematite

Your String.Split example seems like more of an indictment of the language/library design than the GC. For instance, the language could compile uses of an array literal to a static value that's reused across invocations.

> This kind of thing is where I really hate the big GC systems, the JVM and the CLR.

All the evidence points to the problem being sloppy programming and nothing to do with the runtime.

Whoa. I'm assuming I'm not doing enough String.Splits to worry about performance, but what's my alternative? Looping through the string myself?

That's just one example. There's plenty of APIs that force garbage to be created for no good reason. For string.split what I've done when it was critical, is to statically allocate an array for each type of split.

For other APIs, I'd create a state object for each request or piece of work that contains assorted buffers and other temp objects, then pass it around as needed. Ugly, but at high processing rates, every allocation counts.

There can be two allocations when you call String.Split, one at the calling site where you create the array of delimiters to pass in, and one inside that creates the String[] to return.

The parent refers to the first, which you can avoid by creating your char[] of delimiter[s] once and reusing it.

Some people have built string processing libraries which operate on struct which have a string object reference and an index to the first character and a length. This lets you slice etc the strings without forcing any heap allocations.

Your comment is absolutely adorable, for anyone writing C code. :-)