Go, don't collect my garbage
blog.cloudflare.com
blog.cloudflare.com
Any language/runtime will support just allocating a large blob of data and then playing with it with code that looks like C. Effectively once you want to write perf sensitive code in a garbage collectied language you have to stop writing idiomatic code for that language.
That can mean using data orientation (SoA for example is a very non-idiomatic thing to do in most C-like languages and especially OO ones). Not using heap allocation at all for any length of time is definitely non-idiomatic in Java/C#/Python/js etc., but that's what you need to do if you want any kind of performance.
There are two truths here: 1) Any language can be as fast as C and 2) When they are as fast as C, they also look like C regardless what it was to begin with.
Only on GC languages that don't offer proper language features for value types.
Sadly most of them failed to gain market share, like Modula-3.
Eiffel and D are some of few around that offer such capabilities, and C# has been getting some love from System C# features, with spans, new ref types, and GC regions.
Using the low level features in C# (in the past structs, preferably in an AoS layout, then Span<T> etc tomorrow) will effectively make it a "Safe C". When you use those, you can't use idiomatic C (linq, heap allocs etc). You effectively agree to use a completely different subset of the language for your high perf tasks, and when you do, it doesn't look like the C# you have in the rest of your app.
Which is why in most languages, you have to effectively write C (in that language). Not even sure if it's possible in (C)Python, but in C# it effectively means using a lot of "unchecked" blocks, pointers, etc, obviously skipping any interfaces or other vtable lookups etc. The C# you end up with (only structs and no classes, no heap allocation, lots of raw pointers, lots of unchecked and unsafe etc ) will resemble C a lot
In a language like C that isn't an option, and 100% of the code might be unsafe thanks to UB.
Having said that good lockless implementations along with good use of the right data structures can eliminate some or all of the data bus overhead (depending on the data types and uses). RCU[0] is one such example of single object immutability that in effect allows for mutable data structures that can handle multi-threaded environments at scale.
No compiler has precognition or is directly cabled into the developers brain (Though I wish it was) to tell what they want to express.
The compiler might not be able to tell a preallocation is worth the effort if it cannot determine how often a function is called. Preallocs waste some startup time on the first time compared to dynamically allocating on the stack or heap and throwing away afterwards and also leaks memory.
The compiler would have to be able to tell wether or not a function will be called often and is timing sensitive or whether it is called rarely and latency doesn't matter or any of the millions of variation in the spectrum between those two extremes.
A perfect compiler would have to (essentially) solve the halting problem by being able to tell how programs will behave all the time.
AKA the real "sudo make me a better facebook".
Some programming languages permit different implementations to behave differently (argument evaluation order in C for example), but natural language is a whole different league. What you describe would be an extremely impressive AI system, but I wouldn't call it a compiler.
There's an XKCD for everything: https://xkcd.com/568/
https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Just-in-time_compilation https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_pr...
Could you provide an example?
… Unless your language offers appropriate zero-overhead abstractions. C++, for all its flaws, does this extremely well. Then again, C++ is of course (usually) not garbage collected. The problem with the “write like C” approach is that it forces you to write low-level code, and that is usually messy, not self-documenting, and doesn’t take advantage of advanced type system features (which affects correctness).
I think it was ryg who said on twitter that these abstractions are not really "zero cost". C++ abstractions get you in a mindset that might be detrimental to performance.
> The problem with the “write like C” approach is that it forces you to write low-level code
No, you have to think in what order to do things. Yes, you have to write low-level code, but not all the time. Ideally you write the low-level code in one place and the high-level code in another place.
If you do allocations in your high-level code (which is admittedly what you see in many code bases), you are doing it wrong. (speaking of separation of concerns, for example). This has ramifications on maintainability as well as performance.
I'm currently writing a little game engine, where I explore an architecture where these things are neatly separated out. (coincidentally, most allocation is done at compilation time, and much "code" is done by the linker).
There is of course still a place for script-style code that lacks an elaborate architecture. And I guess C++ is the right choice to do that if you need "tight loop performance" or need to interface with C code.
That certainly depends on the application but having been involved with several high-performance libraries written in C++, I can say confidently that C++ abstractions are largely beneficial. For instance, you can often control and avoid memory allocations (but see below): in particular, if you write algorithms interfaces in the style of the standard library, your code will perform very few, if any, unnecessary allocations, and yet be fairly high-level.
> Yes, you have to write low-level code, but not all the time.
I was talking specifically in the context of writing low-level C-like code, as discussed by my parent comment. You wouldn’t write “high-level C code” outside of C, there’s no need.
> If you do allocations in your high-level code you are doing it wrong.
As a blanket statement, I don’t agree with this in C++. If you’re careful you can perform (even dynamic!) allocations in C++. Just make sure that you dispatch to a pool allocator. But of course I agree that this requires care, and lots of code should avoid them.
Not sure about that - if you want performance you might need to stick to the C way anyway, so maybe you can't take advantage of programming in a scripting language. I've heard of some cases where game programmers ported their game logic back from their script language to C/C++.
> For instance, you can often control and avoid memory allocations (but see below): in particular, if you write algorithms interfaces in the style of the standard library, your code will perform very few, if any, unnecessary allocations, and yet be fairly high-level.
Another problem with that approach is binary compatibility. Let's take the infamous std::string as an example (not even a templated class). The implementation is much more likely to change than e.g. const char pointer plus optional length, since also memory management and whatnot is needed.
If the implementation changes you are in trouble everywhere, including most of the places where a pointer and size would have been just fine (because no allocation is needed there).
That just happened to a coworker who now needs to setup a another system and build new binaries on it to support a customer's "old" platform (Ubuntu 14.04)
As long as all the abstractions and language features are compile-time evaluated to what people may consider to be "C-like code", then it's all the same.
I don't like the term "C-like code", though. It's not C's fault that fast code looks like that, and it is certainly possible to write "prettier" C code than what people have in mind.
From what I’ve heard you’re almost certainly right. I just don’t know enough about Rust to comment on it (unlike C++).
But it's not completely 'pure' on that front.
LLVM's coding standards prohibit RTTI and exceptions precisely because they aren't zero-overhead.
https://llvm.org/docs/CodingStandards.html#do-not-use-rtti-o...
What C++ needs is a way to specialize types so that they automatically know if they need heap or stack storage without the programmer having to be explicit.
C++ doesn't completely free you from considering whether a type allocates memory on the stack or on the heap, but it does let you limit which code needs to concern itself with where values are allocated to a very small portion of the code. Most code written to work against a std::vector will also work against a std::array unless it is relying on the ability of a std::vector to dynamically resize.
You’re completely right but the way this is phrased shows a common misconception about C++/RAII: it was never about freeing the programmer from considering how to manage memory. Rather, it was about automating the mechanics behind these considerations, and adding a layer of type safety (which C++11 tremendously improved).
If you don’t want to think about memory management, use a garbage collected language (but accept that it doesn’t scale to all scenarios). C++ forces you to think about resource management, it just makes the implementation of resource management vastly easier than C.
Isn't it rather that switching between SoA and AoS is tedious? SOA style itself (using common indices into multiple tables) works ok in C, IMO.
Which really defeats the point of the garbage collector, because the good ones do that kind of work for you! The good garbage collectors re-use pre-allocated ranges and pools of data.
Garbage collection usually works out to a performance / latency tradeoff. (A GC app runs faster than a non-GC app, but has latency.) If I get to the point where latency, even with a pause-free of concurrent GC is a problem, then I know I chose the wrong language.
I think there are several layers to this. First of all, GC'd data is on the heap, and often you are already doomed if you do stuff (in spread out places) on the heap at all because of cache issues.
Secondly, if we consider "normal" applications that have small parts of perf sensitive code, while a majority of the codebase is not (such as e.g. a 3D rendering application where you hit a button and it goes of and renders an image for 1 minute), then you'd have plenty of time to set up your data, and then you just "operate" on it for the critical part of time. The same isn't true in all scenarios.
Same thing with e.g. a game engine. It's fine to allocate massive amounts of memory, but then in each frame when you have a 16ms budget you allocate nothing. Because the delay loading a level doesn't matter whether it's 5 sec or 10, but you can't spend 17ms in a frame instead of 16ms.
Another problem (and benefit) in most GC'd languages is that the GC is there for developer convenience, and the class libraries know its there so all standard library features (such as just iterating over a collection of items) risks creating new heap allocated objects, which might trigger new garbage collections. In e.g. C# I can't make a standard list with a custom allocator. And I can't "foreach" over something and be sure I didnt' allocate etc.
> A GC app runs faster than a non-GC app, but has latency
I didn't mean to compare an app that does "new Foo()" in the inner loop with one that does "malloc(...)" in the inner loop, I was trying to make the point that you can't do either in the critical part.
> then I know I chose the wrong language.
Right, but the thing is you might have chosen the perfect language for 90% of your code. The language choice might be too late to change and you just need that concurrent inner loop (rendering loop, encryption pass, whatever) to be quick, and the solution is a lot of time to just allocate nothing.
Most applications don't require tuning and have acceptable performance for real world usage. The reason people write in GC languages is because malloc() and free() have error cases you can avoid with a GC. A GC is a well tested structure and less likely to leak memory by forgetting a free() or doing a use-after-free.
Of course we have static analysis tools for this but those aren't perfect.
There are perfectly legit reasons to use GC, one of them being not having to do malloc()/free() at the cost of some performance characteristica which can be "tuned" as you describe it.
That is not always true. Also, preallocation can be impossible (due to limit on heap size) if there is no way to free memory without the GC running. Of course, you're right that there are tradeoffs, the only argument here is about the magnitude of the tradeoffs - the OP shows a hidden cost of GC that is not talked about often enough.
I would love to see a benchmark of GC vs non-GC using the same compiler and language for an accurate comparison. Probably also GC vs non-GC naive vs non-GC optimized.
I think you mean "throughput / latency"? Latency itself is a measure of performance.
It's a fair bit more complicated than that once you start tuning the GC, because the GC can be tuned towards either throughput or latency, and it can be tuned aggressively towards either end. For example, for a given program you can tune the GC so you get fantastic throughput for batch programs. Use a copying GC, and set the heap size large enough that the percentage of live pages is very low when you collect. On the other hand, you can use something like the tri-color collector, which runs in parallel with the main thread and kills throughput, but has minimal impact on latency (sub-millisecond pauses).
For context, I'm a game developer working in Unity. We do basically all of our dev in C#, which is a language I love. But because it's garbage collected, and you can see performance hits when the GC runs. In VR dev especially (targeting 90 FPS) this is unacceptable.
So we profile, look to see where we're allocating anything onto the heap and figure out a way to cache it or re-implement it in a way that makes no garbage. We recently shipped a mobile VR experience and one of the best parts was being able to profile my gameplay code and seeing the '0B's next to its allocations every frame.
I understand that this may not work for everyone, and I fully believe that CloudFlare's got some talented engineers on the team, but there are ways to think about and structure programs in a C-like fashion so that you're more cognizant of the memory you are allocating and using from the heap.
I care what you have to say; that you came here to say it is irrelevant.
https://news.ycombinator.com/item?id=15683745 https://news.ycombinator.com/item?id=15590466 https://news.ycombinator.com/item?id=13614442 https://news.ycombinator.com/item?id=14306868 https://news.ycombinator.com/item?id=14327156 https://news.ycombinator.com/item?id=14532140 https://news.ycombinator.com/item?id=14531421
Sure, their first sentence is irrelevant, but it's forgivable since they elaborate beyond it.
I realize you're joking, but I didn't think I was. It's not like I said "HN is no place for this type of comment". I just stated my opinion
per my knowledge this is not even possible in C#, I mean you can drop to c way easier there or use (stackalloc) (same as Unsafe usages in Java) But I doubt that is not a good idea?
In C#, anything instantiated as part of a class goes onto the heap. Value types that are declared and used only within the function are allocated onto the stack. For example:
public class Example
{
public int a; // Allocated onto the heap when Example is instanced
public int b; // Allocated onto the heap when Example is instanced
public int c; // Allocated onto the heap when Example is instanced
public void CycleValues()
{
int temp = a; // Allocated onto the stack when function is called
a = b; // a, b, and c are already allocated
b = c;
c = temp;
// temp goes away when the function finishes executing
}
public int SumPositiveValues()
{
List<int> positiveList = new List(); // Not a value type: allocates onto the heap
if (a > 0)
valueList.Add(a);
if (b > 0)
valueList.Add(b);
if (c > 0)
valueList.Add(c);
int sum = 0; // Allocated onto the stack
foreach (int i in valueList)
{
sum += i;
}
return sum;
// sum goes away when the function finishes executing
// positiveList was allocated onto the heap, so it persists (although inaccessible) until garbage collection
}
}
There are bigger convos to be had about data architecture here, but now's neither time nor place for it.
I can't share any of my own code, but the sort of things that I'm doing can be exemplified by these two functions from Unity's physics API: [1][2].
The first returns a newly allocated array of the data, while the second takes, as a parameter, the array to populate and returns only the number of elements that it populated. If I know I'm going to be doing a big raycast each frame, I can instantiate the output array to a reasonable size in my class declaration not have to pay the price of the garbage created by allocating a new array every frame.
[1] https://docs.unity3d.com/ScriptReference/Physics.RaycastAll....
[2] https://docs.unity3d.com/ScriptReference/Physics.RaycastNonA...
open Batteries
let bench name fn =
let tstart = Unix.gettimeofday () in
let result = fn () in
let tend = Unix.gettimeofday () in
Printf.printf "%-20s %.3f seconds\n" name (tend -. tstart);
result
let iterations = 100000
let count = 1000
let insert_list_test () =
let result = ref 0 in
for i = 1 to iterations do
let work = ref [] in
for j = 1 to count do
work := j :: !work
done;
work := List.rev !work;
result := !result + List.length !work
done;
!result
let insert_list_test_rec () =
let rec build_list i n acc =
if i > n then acc
else build_list (i+1) n (i :: acc)
in
Enum.map
(fun _ -> build_list 1 count [] |> List.rev |> List.length)
(1--iterations)
|> Enum.fold (+) 0
let insert_dynarray_test () =
let result = ref 0 in
for i = 1 to iterations do
let work = DynArray.create () in
for j = 1 to count do
DynArray.add work j
done;
result := !result + DynArray.length work
done;
!result
let main () =
let x1 = bench "linked lists" insert_list_test in
let x2 = bench "linked lists (rec)" insert_list_test_rec in
let x3 = bench "dynamic arrays" insert_dynarray_test in
assert (x1 = x2 && x2 = x3)
let () = main ()
linked lists 0.648 seconds
linked lists (rec) 0.657 seconds
dynamic arrays 1.295 seconds
The problem that Go has here that it's GC is neither generational nor compacting, so it can't do that, so there's a fairly high constant overhead per allocation, whereas OCaml's or the JVM's temporary allocations are only marginally more expensive than alloca().
Note that there can be good engineering reasons to have a non-generational, non-compacting GC (for starters, compacting GCs make C interoperability more difficult). So, yes, if you have such a GC, you do have to watch out for avoiding extraneous allocations.
The problem is that often you don't know exactly how much memory you'll preallocate. This is where in (say) C++, you'll resort to an arena or pool allocator; a GC with a bump allocator will give you what is essentially a smart arena allocator by default for your temporary allocations.
let insert_array_test () =
let result = ref 0 in
for i = 1 to iterations do
let work = Array.make count 0 in
for j = 1 to count do
Array.set work (j - 1) j
done;
result := !result + Array.length work
done;
!result
# main ();;
linked lists 2.983 seconds
linked lists (rec) 2.827 seconds
dynamic arrays 5.152 seconds
static arrays 1.718 secondsFor example, note how the `ref 0` in the code does not actually incur overhead; the compiler does escape analysis and will stuff the counter in a register or stack location.
When I was writing numeric stuff in common lisp, the performance critical stuff all ended up looking pretty c-like, and as far as I can see it's similar in OCaml (for really high performance stuff).
It's not a bad trade off, especially for anything following the 80:20 heuristic. That way 80% of your code can be written more expressively and less error prone.
Agreed. If you want a high-performance systems programming language, then you generally want one that allows you to transparently write code for GCed references, untraced references, and pass-by-reference parameters. Nim and Modula-3 come to mind as languages that support this, to some degree or another.
> and as far as I can see it's similar in OCaml (for really high performance stuff).
The big problem that OCaml has with numerics is that it boxes floats by default. The compiler does a lot of work to avoid that boxing where possible, but it's not always possible, even in common use cases. (The JVM shouldn't have that particular issue, though.) In this case, a fast allocator can at most mitigate the overhead associated with boxing.
My point is also not that it's always better, but that it's not safe to make blanket assumptions about GC overhead.
And if you're dealing with long-lived data of varying sizes, then the manual allocation story isn't that great, either.
Hence the maxim:
"The determined Real Programmer can write FORTRAN programs in any language."
It applies to C as well.
My first large web app was old enough that we did it in C++ as CGI...
One of the first performance optimizations we did was accept that most object lifetimes roughly coincided with the end of the request, so we simply didn't deallocate anything and let the OS just free the whole thing at once. You can do the same in most GC'd languages by just tuning GC limits.
(the second performance optimization we did was to to statically link everything; loading a static binary that's constantly in the cache is/was very fast; and it was enough to make CGI surprisingly competitive in terms of performance)
Preallocation certainly has its place too, but in this case we tried custom allocators and it simply didn't make much difference. So before rewriting your code, see if you can simply turn off deallocations/GC without too much memory pressure.
I did actually try a custom allocator that'd just hand out sequential pieces of memory from a large buffer without keeping track of it at all, but if it made things any better, the effect was small enough that we couldn't reliably measure it.
Some will try too hard and be "too smart" and will add extra cost, but most will try to get out of your way until/unless there are deallocations leaving holes.
They'll typically ask for a multiple of pages at a time, and some increase the number of pages requested each time. A quick check with strace right now shows modern glibc in my instance allocating about 2MB at the time when running "find" for example. It will do more if you ask for larger chunks of memory at the time, of course.
In term of fitting in, some allocators will segment the heap it requests into separate free lists, but many will do this first on deallocations to combat fragmentation. E.g. here's a simple example that shows objects linearly allocated on the heap (on my system, anyway) despite different sizes:
int main () {
void * a, * b, *c, *d;
a = malloc(4);
b = malloc(8);
c = malloc(12);
d = malloc(16);
printf("a=%p\nb=%p\nc=%p\nd=%p\n", a,b,c,d);
}
$ ./test
a=0x1a7d010
b=0x1a7d030
c=0x1a7d050
d=0x1a7d070
$ ./test
a=0x1678010
b=0x1678030
c=0x1678050
d=0x16780a0
In this case, the cost of doing the allocation typically boils down to seeing if there is space in one of the free lists, then if there's space at the high water mark of the current heap allocation, and only if there is not do another system call. When there is space, the cost tends to be a few comparisons, adding some book-keeping details, and increasing a pointer. It's extremely rare to even show up when profiling.
So the cost of those syscalls tends to get amortized over enough allocations that the cost is negligible. E.g. with my example above, glibc does a single allocation to cover the 4 allocations I did.
For a process that only ever allocates, the number of system calls will depend on how aggressive the allocator is about growing the size of the extra memory it requests, but in general the number of syscalls will be small.
I appreciate the attention to syscalls - it's a pet peeve of mine that people tend to think library calls are cheap, without realising the cost difference between a mostly user-space one vs. ones that do a syscall and hence context switch every time (read() and write() are particularly persistently abused - add buffering, people), but malloc() or C++ new are rarely amongst them (doesn't mean you should do lots of tiny ones, though; if you do, profile and consider a pool allocator).
This'd speed up development but wouldn't help performance with CGI, since spawning a process is very expensive. This is why CGI is dead.
Actually, in reality things were exactly opposite of what you suggest: it provided no noticeable benefits for development, but massively improved performance in production.
Spawning a process on Linux is more expensive than spawning a thread or reusing the same process via a fastcgi type setup, but it's not all that expensive when you're spawning a statically linked app with no complex startup code and the app is guaranteed to be in the buffer cache at any moment.
What killed CGI for most uses was not process creation overhead, but dynamic linking and applications not written to take advantage of the fact they'd die quickly.
We knew that because we actually measured first, before deciding what strategy to take.
We got performance very close to a fastcgi type setup with the above method, and without the complexity of ensuring we didn't have any resource leaks anywhere.
Doing the above was enough to bring process startup overhead sufficiently far down our list of performance issues that it was not worth optimizing further.
Most or all C and C++ standard libraries come with generic allocators that work OK in most scenarios, but not optimally for specific scenarios.
This is a very synthetic workload, and all allocation were done by the standard library.
Therefore, I read his article with the intent of placing his findings in the "taxonomy" of GC tradeoffs.
In that vein, I noticed that his benchmarks and graphs do not include measurements for memory footprint. Instead, it has throughput statistics such as "ops/sec" or "sign/second".
I think I see why working memory size isn't emphasized. He's testing a crypto algorithm "ECDSA-P256 Sign". I'm not familiar with it but I assume we can think of it as a typical hash function[1] that doesn't require unpredictable dynamic allocation of memory as the algorithm processes the bytes of input. (A typical hash function uses the same fixed amount of memory whether hashing 1 kilobyte or 1 terabyte.)
If his synthetic benchmark to play with GOGC was using a different type of workload... such as a text parser that stressed the GC with thousands of different sized memory objects, we'd see memory footprint as part of the analysis.
[1] https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signatu...
Also, Go crypto makes a heavy use of interface that also make stuff escape to the heap. Of such things are a bottleneck, it might be worthwhile to "un-interface" them.
I believe Go does something like set a threshold of memory use. Once that memory usage is hit, a collection is run. After that collection is run, it uses the new accurate amount of memory used to set the next threshold. The threshold of course start small and have an exponential increase of some sort.
When the program first starts, the threshold is set fairly low. If you immediately start allocating lots of things and keeping them, a common case, it grows to a sensible size in a reasonable number of allocations and sweeps (which as you are growing into your initial footprint, amortize out pretty cheap), and in a lot of cases you won't think much about the GC until you are really pushing performance.
If, however, you write a program that isn't holding on to much RAM in the steady state, such as a crypto algorithm (which for all their algorithmic complexity typically don't take much RAM to run, as compared to, say, loading an image, and I don't know for sure but it's possible this ends up all stack-allocated), and that algorithm allocates a lot of short-term stuff, perhaps even just allocates for its final answer, then you get into a situation where you are quickly generating enough garbage to cross the initial low threshold, but the GC immediately cleans it all up when it runs, so when it is deciding on what the new threshold should be it still picks a low number, causing it to repeat the cycle indefinitely.
Fixes will immediately leap to your mind, but remember you must evaluate your fixes holistically, not as if this is the only case in the world. If it were the only case in the world, it would be an easy problem, and Go never would have exhibited the problem in the first place. In particular, as you consider your "fix", consider also that a real program that used real crypto would be much less likely to see this pessimal case occur, because it is very likely the rest of the program would push the memory thresholds up to something sensible, so the fix of "do nothing" is actually quite likely to work well on non-artifical-benchmark programs. It is likely that the most obvious fix that comes to mind will result in making a program that is deliberately running some small server process or something appearing to leak memory after running for a long period of time, for instance.
(I suspect quite strongly in this case that the correct "fix" is to do nothing.)
What about a fix that gives the user optional control over the thresholds, or even just a hint as to the desired behavior? Keep the defaults as they are, but add a “pragma” that would set the minimum threshold for GC? Or it could go a long way just to have an option to set a minimum time between collections. While it could easily make things worse for certain situations, I much prefer the option to hang myself over an immutable default that periodically does so without my input or consent.
Isn't this "highly concurrent" the main selling point of Go?
As it happens [lol] facts asserted themselves and (at least some in) Go community saw the reason as to why "communicating by sharing" paradigm ever got traction. "It's performance, stupid", to paraphrase Slick Willy, not because other language designers were thoughtless. So now it is "idiomatic" to see Go code using both paradigms (locking & passing channels). The original matra was quietly dropped along "systems programming PL" bit somewhere along the line.
This OP type of article, in an interesting way, continues this pattern of "let's do it this [obvious] way" in face of dealing with concurrency. No less an authority than Cliff Click has asserted that lock-free type algorithms pretty much require a GC.
Concurrency and memory management seem to be zero-sum games. Root cause is clearly Physics, and not the mechanism devised to deal with the physical realities.
I don't think the article says anything about concurrent allocations; in fact, the various signers don't need to share anything (mutable) to sign the requests they get.
And? Did I say they did?
> I don't think the article says anything about concurrent allocations
No it doesn't. Point was that it is not a general purpose approach. GCs are.
The "original mantra" is "Channels orchestrate, locks serialize" and hasn't been dropped by anyone.
There is nothing wrong with that approach. It does put a ceiling on performance, however.
If I understand the article, the individual goroutines aren't* communicating; they simply compute a cryptographic hash. However, they are communicating by allocating memory for the hash and then discarding it immediately.
From this, I think we can learn that this implementation has a single-threaded, non-concurrent garbage collector, and the highly-concurrent goroutines are hammering the snot out of it.
The question not answered (as it wasn't asked) is how many workload profiles can be catered for? But as a sample of one, this appears successful.
The important question is whether or not they can get the scaling with some setting.
Also there is no data provided to show how bad is Go's GC compare to Java
If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system, and all this additional metadata has to be juggled. If allocating new memory is a system call, that has costs. If you get lots of page faults because you keep using new pages, that has costs as well.
> If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system ...If you tell the Go runtime not to garbage collect but keep allocating memory, that memory has to come from somewhere. That somewhere is the operating system.
If your program is far from exhausting the RAM but is fully utilizing the CPU, you might want it to save CPU by using more RAM, equivalent to increasing GOGC here. The rub is that it's hard for the runtime ever be sure what the humans want without your input: maybe this specific Go program is a little utility that you'd really like to use no more RAM than must to avoid interference with more important procs. Or maybe it's supposed to be the only large process on the machine, and should use a large chunk of all available RAM. Or it's in between, if there are five or six servers (or instances of a service) sharing a box. You can imagine heuristic controls that override GOGC in corner cases (e.g., assume it's always OK to use 1% of system memory), or even a separate knob for max RAM use you could use in place of GOGC. But the Go folks tend to want to keep things simple, so right now you sometimes have to play with GOGC values to get the behavior you want.
"We already used Go" is a totally valid reason. "It's rare (or as you say potentially never an issue) that we actually need to avoid GC" is another.
(Cloudflare engineer but I have nothing to do with the recent blog posts)
I've seen a lot of advocacy over the years like that which has seemed to inspire more backlash than adoption. I generally prefer something constructive in the tradition of showing a better way to do something — i.e. in this case, perhaps show a Rust crypto library which delivers better performance so people could judge whether that kind of thing is enough of a draw to be worth migrating or, in the case of Rust, taking advantage of its embedding characteristics.
“why is Rust not chosen as main language” seemed like an unnecessarily accusatory way to express that idea, especially given that language choices are usually driven by a number of factors rather than a single benchmark.
Go has a history of doing the simplest thing that covers the majority of cases, inherited from Plan 9. For one thing, although I don't know its current GC story, its original collector was very simple.
I wonder what would happen if you transplanted a recent collector from Java, where the strategy is to optimize the crap out of anything that moves, into Go?
Obviously, it wouldn't be as fast as not allocating, or at least not collecting, in hot code, but what would be the actual penalty?
On the other hand, what about, say, Pony and its per-thread collectors?
The current official Go runtime performs very very poorly when virtual memory is involved. It may make the while OS non-interactive so that you must restart your computer.
A SetGCMemoryThreshold(maxMemory int) opition is better for many programs.
Just adding some heuristics to override GOGC's behavior in extreme cases could help, e.g. assume by default OK to use 1% of system RAM and not OK to use >75% of it regardless of live data. I'm not sure we can realistically hope to get that--adds complexity, folks dislike arbitrary thresholds, and any change will break users who have optimized around current behavior. It might be you could approximate the desired behavior for reasonably well-behaved programs from a third-party package using runtime methods and a timer, but that's a drag.
[1] especially when comparing to Java when it comes to GC customization.
As another comment in this thread explained, GCs have CPU and Memory tradeoffs and you can easily make a GC that would work excellent in the OP benchmark but would suffer severely under other workloads.
There are lots of good criticisms of Go to be had, but it's runtime is pretty remarkable, in my opinion.
Or maybe not - maybe that one knob really can solve all the problems. But I really was surprised that anyone would ever need to actively configure a GC for such a simple use case, so I've learnt something at least.
Actually, GC == automatic memory management, just not automatic performant memory management
Well, it seems we don't have a simple GC that performs well in all cases, and really why would we expect to? So choosing simplicity means leaving performance on the table, at least for now, and while there are languages that do that (Python) it doesn't seem like a good fit for Go.
We don't appear to be much better off than we were with C. I guess the server melting is probably preferable to an exploitable smashed stack, but prevention of both issues appears to be to hire geniuses who don't make mistakes while doing enormously mistake-prone work.
"Everyone gets correctness, you need geniuses to get maximum performance" is a huge improvement over the reverse.
I was most impressed that he managed to get very close to perfect scalability by twiddling the one knob on offer.
Java GC customisation is an art. This analysis was entirely mechanical. If the author wasn't just lucky i.e. that this is sufficient for a large number of workload profiles, then that's very good news.
That said, there are exceptions. I don't think GC is something simple enough to be an exception, but those exist.
I expect Go's current behaviour is right for memory-intensive programs, where "limit the wasted memory as a ratio of the actual working set" is a reasonable high-level goal. For a CPU intensive program you might have preferred a "max x% CPU overhead" knob, but just a minimum memory size would work, as this is mostly an artifact of the very small memory usage of benchmark program.
Java's G1 has "max pause delay" as their preferred knob, but Go has stuck that knob on low since around version 1.6, which is a good choice for what is essentially a language for network services.
I am surprised that the difference is so great - and that the GC seems to think it's a good idea to keep collecting 4 MB of data at a time - but I guess the Go team have been tuning heavily for interactive performance i.e. small pauses.
I guess in a situation like this you'd prefer something more like 'server' GC - maybe wait until you've allocated 1 GB then run through and collect everything you can.
In games, I wonder if you'd like to turn the GC off during frame rendering then collect while presenting the frame/waiting for VSync.
For soft real-time audio - synthesizing a few samples at a time - what would you do? OpenAL wants you to queue up thousands or tens of thousands of samples, but that sucks if you need to react very quickly.
If anyone has any ideas, please say. I'd love to do soft real-time audio in a thread even in e.g. C#. (Edit: Maybe JIT some code that doesn't allocate? Or even have a separate interpreter written in C that has a constant-ish upper performance bound & no malloc.)
In fact, I do - daily. I'm not stating that this exact issue is the key reason, but because C is very simple with not much of an abstraction layer between software and hardware, where most such (unexpected) performance issues arises. Yes, it might be a single flag to makes things run normal, but it might also take 1 week in production to find such "magic" bugs.