Hacker News Reader

Comment by hu3 | Hacker News Reader

How can D use so little memory despite sporting a GC?

    Go Memory usage: 13mb
    Rust Memory usage: 9mb
    D Memory usage: 7mb

View on HN

Another answer for this particular case is that the D program has less features. Quoting the blogpost:

> Rust downloaded a lot of crates, but that's probably because the Rust version of the code has more features than mine has. For example, it fetched rpassword, a tool that hides password characters as you type them into a terminal, much like Python's getpass function. That's one of the many things I don't have in my code.

Thanks, that could be part of it. I added some of the missing features, namely colored terminal output and getpass (thanks to the reply above), but the memory usage didn't fluctuate much. The D version is still missing websocket support for the "watch" command, but none of the clients use it in the benchmark.

What do you think a GC does? It will use more memory but the design of the library using it pays a huge role. D's standard library is intended to be as lazy and efficient as possible within its API (i.e. most ranges are lazy and work on the stack to start with)

Also, quick tip for anyone writing scripts with a GC or malloc/free: turn the collections/free off and just allocate until you finish the script. The OS collects the memory and you save a few seconds.

I agree that this is helpful sometimes, but reading this causes flashbacks to deleting files and killing processes manually while watching htop, hoping that a serveral-hour-long process won't die in its last few minutes

Oh yeah. Definitely small batch jobs only, it's a good trick for parsing too if you know (MEASURED) it's a small % of the task but very memory intensive or fucking the cache up.

How can D use so little memory despite sporting a GC?

My guess is that you have experience with platforms such as the JVM or CLR (dot NET) and expect garbage collected languages to use up a lot of memory.

The JVM and CLR use generational/copying GC. These types of GC sacrifice memory to get efficient collection -- the more memory they use, the more efficient the GC [1].

Both Go and D use non-copying GC which has a lower memory overhead.

I am not sure about D, but Go also uses escape analysis so that data structures are allocated on the stack instead of the heap when possible which can help to reduce garbage and hence memory usage.

[1] The principle is similar to the one described in this paper where the GC time is independent of the amount of garbage: https://scholar.google.com/scholar?cluster=48058747733581794...

> I am not sure about D, but Go also uses escape analysis so that data structures are allocated on the stack instead of the heap when possible which can help to reduce garbage and hence memory usage.

D sometimes does this but for the most part it depends on how you write the code. strings, large arrays, and class objects are typically GC'd but struct instances and small arrays are often not.

The code in the blog post appears to mostly use small stack memory.

The only weird thing i see in the blog post is 30 second compile. Yikes, this should be more like 3 second from scratch, max - it must use some heavier-than-necessary dependencies.

It seems to include botan, which is a fairly large C++ library. You can be assured that by far most of those 30 seconds are linking. The compile step itself for D is going to be done in <1s for the code shown.

Lisp had a GC in the 1960s when 7mb of disk was hard to come by, let alone ram.

Lisp also had a reputation for wasting memory. The joke, from back when 4MB was still a lot of RAM, was that emacs is an acronym for "eight megabytes, all continuously swapping."

Being around for longer means more optimizations

D has been around for longer but Rust and Go both have (I'd guess) on the order of 100 full time staff each. D has something like 3 at most.

What? Rust has less than 5 full time paid devs. And with the fall of Mozilla it's shrinking. But yeah external contributors summed up are a big human resource

But how many companies outside Mozilla use rust in a capacity to contribute to the compiler, standard library, and crate ecosystem? e.g. Now rust is in chromium if Google want stuff fixing they can pay for it to be done in the same way that Linux is developed mainly by 9-5ers not working directly for the Linux project specifically. D has no paid developers at all full time IIRC (Walter Bright is living off his empire empire I think).

Go has Google's direct backing, which is more what I was pointing towards.

According to the recent blog post that was on the front page, Chromium is still exploring the possibility of using Rust. They’ve yet to commit.

I guess it's a case of quality over quantity.

D developers are all experts in the art of crafting compilers and language design.

9 women can't make a baby in a month

As a occasional contributor I can attest that most people on the D forums/GitHub are very sharp, although (I do not like Go) Rust's team seem excellent too (albeit slightly more academic?)

The rustc developers are excellent as well.

Never said the contrary.

Imagine what they can do in 19 years.

Go GC has some smart people working on it too.

And Go, by the virtue of having less features, has to allocate quite more. Eg: Go doesn't have lazy ranges / streams. Dynamic dispatch using interfaces has to allocate memory on heap.

Also, I expect D standard library to be optimized for Non-GC use case at least a part of it, because that's one of its use cases.

Compiler's escape analysis matters too. I'm told Go's escape analysis is not very sophisticated.

Last, it may be case of missing features in D version as a comment mentioned.

Post-hoc: I'd say that it's easier to communicate between 3 people than between 100. And, so, it's more likely that 3 people will be able to maintain a system where they each have a pretty clear understanding of the whole thing.

I would not have predicted D would be better than Rust in any measure if those are the number of maintainers.

I'd guess that the larger team must be able to do more. "Faster alone, further together".

Reply on news.ycombinator.com