GC Fun at Twitch
modularfinance.se
modularfinance.se
This proposed alternative of just toggling the GC on/off outright in a sleeping loop feels like a pretty big sledgehammer - and just as much of a hack. The 500ms sleeps are enough to see 5 GC cycles, going off of the original twitch blog's 10 GCs/second numbers, which would also concern me - as a potentially unwanted latency spike. I'm also curious what happens when the GC is toggled back off mid-GC. It's more code, and feels brittle. That ReadMemStats sync point may be worse than the GC spam in the first place!
The trade-off of the knob-less approach of Go's GC I suppose.
This is unlike the golang gc, which is tuned for latency at the expense of throughput, with no way of modifying its behavior without resorting to hacks like the article in the post.
[1] https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...
The method presented in the article does seem better in that it is using well-known and documented parts of Go's runtime api, but I think it might be problematic for other reasons. Fiddling with GC behavior is always a little risky because it works fine until you hit some weird corner case and it blows up.
For example - What happens if that goroutine doesn't run for longer than you expect and you leave GC turned off while another goroutine is creating a ton of garbage? Might never be a problem, but it depends on allocation behavior and how much headroom you have.
So it feels more correct, but also seems like it requires a lot more tuning and testing to feel confident about it.
Sort of. A change in the undocumented behavior might cause you to lose your fine-tuning at some point in the future, but I wouldn't say it'll ever cause it to break. You're just telling Go how much memory you want to pre-allocate. It'll continue doing that; if that stops getting you the same GC benefits you wanted, then at worst you'll be back in the same boat you were originally.
Writing your own GC routine, on the other hand, gives you a ton of new opportunities for introducing very real breakage via your own code.
In the issue thread Caleb Spare also proposed a minimum heap size so that you get GOGC-ish behavior once your app uses enough RAM, but don't have constant GCs with a tiny heap.
There's definitely a common issue where the GOGC heuristic doesn't take advantage of situations where it can collect less often but still remain in the "don't care" range of memory use. (CloudFlare talked about the same thing making benchmark results weird: https://blog.cloudflare.com/go-dont-collect-my-garbage/ )
And there can definitely be situations where GC'ing a bit more would be worth it to keep a process under an important memory threshold to avoid swapping or OOM kills.
The designers famously don't want too many knobs, but some other ways to convey user priorities to the runtime could certainly save users from some awkward workarounds and fiddling w/the existing knobs.
What works well is when you calculate your own capacity needs, then just set the autoscaler to change to that new capacity number. In other words, using your knowledge of how your system works, you'll make better decisions than just looking at secondary metrics like resource utilization.
I know I've done manually triggered GC in Ruby and Java but I don't know enough about Go to say if the article's suggestion is reasonable.
There's enough rockets on the rocket-powered horse that is GC to make it to the moon and back.
I'll save this for the next time someone posts something along the lines of "you can't program X in a GC'd language because the GC is so unpredictable".
"Quite a hacky solution" describes every single detail of every scrap of code connected in any way to GC. It is the whole point of the enterprise. If hacky solutions make you unhappy, your only route to happiness is to run very far away.
A lot gets done with very hacky solutions, and you will never need to throw a rock very far to hit somebody who swears by them. Those of us who don't haven't time to get that work done, so for most of the world's work, it's hacks or nothing.
I can't read the referenced twitch article from work so cannot comment. I'm also not sure of the practical loads and implementation details and am surprised that the Go GC was generally an issue to begin with.
I know I've purposely called GC for languages that use it for ETL jobs that run on shared servers to minimize memory usage before.
It is fundamentally misleading to call C++ or Rust "lower-level" languages than Go or Java. (As it is, also, to say "C/C++".) Both Rust and C++ support much more powerful abstractions than either, making them markedly higher-level. That they also enable actually coding abstractions to manage resources (incidentally including memory resources) reduces neither their expressiveness nor the productivity of skilled programmers.
The point of Java and Go is that less-skilled programmers can use them to solve simpler problems more cheaply. Since most problems are simple, those languages have a secure place.
Haskell and Idris and the like (other languages with a type system in the calculus of constructions) inarguably support a higher level of abstraction than Rust does, and are also “GC-obligate” languages. So your example is something of a red herring. I could say the same about Kotlin and Swift and Scala, none of which really have a strong story for static memory safety like Rust has, though it’s being considered for Swift. The only language that is reasonably complete that I could think to compare it to is ATS, which is far more complex as a result.
You can also use something like https://github.com/artichoke/cactusref - which provides an equivalent of Rc<T> with nearly-seamless, timely detection and collection of deallocation cycles. This gives you the equivalent of full GC, but using a "zero-overhead" approach that integrates more cleanly with how Rust idiomatically works.
Two common problems in most reference counting implementations.
A cactusref is owned by a single thread so there's no STW issue, but you also can't share them mutably between threads like structures available in some GCed languages.
I wouldn't suggest that a given language favors more or less skilled programmers. There are plenty of skilled programmers that will choose a given language simply for the time to get things done. Not every problem needs absolute performance and memory scarcity, in fact I would suggest that most don't.
I've used some very low level languages as well as a bit of assembly in the past. All the same, JavaScript is the language I enjoy the most, simply because I can get things done in flexible ways, with many modules already written. There are times where you want absolute performance with minimal memory (embedded systems in particular, though even they're getting pretty powerful) and there are others where you can duct tape something together that only needs to run a couple times a day.
I've seen front end developers that couldn't handle conceptualizing SQL-style data storage... Likewise, I've seen backend developers unable to deal with breaking apart UI components or dealing with event based workflows or managing state outside a database context. I've seen systems developers create the most byzantine, overly complex and buggy systems you can imagine, that don't even work half the time. On the flip side, I've seen aircraft systems designed and built almost entirely in hardware... now some of that is truly impressive (and took years to design and develop, compared to weeks/months many developers will get).
Closer to when I was starting out, it was Visual Basic that was considered the proverbial whipping boy of "beginner" or "less skilled" languages. I've seen very good, and very bad implementations of a great many things in a great many languages over the past few decades. I'd say some of the worst of the worst code I've dealt with has come from the most arrogant people I've worked with. Generally, you aren't as smart or clever as you think you are. And I mean "you" in the colloquial sense. In the end, all anyone (or at least most people/users) really care about is it does the job, and is relatively easy to use.
Except for the heroic efforts from the Rust community, linear types are far from general consumption for any kind of software development.
Plus, having GC does not preclude being able to stack allocate, keep data on manual memory segment, or even resort to manually manage memory in unsafe code blocks.
Examples of GC enabled languages with such features, Modula-3, Mesa/Cedar, Active Oberon, Nim, D, Eiffel, C#, F#, System C# (M#), Sing#, Swift, ParaSail, Chapel.
Eventually Java might get such capabilities if Panama and Valhalla actually end up being part of the official implementation.
Manual memory management is required for some critical code paths, but so is Assembly, both are niches, not something to spend 100% of our coding hours.