Save money over what? It's usually a lot easier to optimize memory use than to rewrite code. Going by the 80/20 rule, 80% of the memory pressure on the GC will be created by 20% of the code. So amortize the price of the rewrite over the expected lifespan of the app. Then compare this to, let's say, the cost of optimizing 10% of the app to eliminate 40% of the memory pressure. Then, also factor in the likely rate of bugs introduced in a rewrite compared to the optimizing GC and the cost of incurring, finding, and fixing those bugs.
Going by this analysis, I would suspect that in some environments where rapid iteration is key, progress is fast, and apps have short lifespans, it might be better to optimize GC instead of rewrite to eliminate it. I'd also expect that in other cases, it is better to rewrite to eliminate GC.
So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.
This might be true in new development. The specific context in this discussion was rewrites.
So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.
Again, you're talking about new development. That's not going to fit everyone's situation.
The problem is that whatever level of GC overhead you start with, or achieve, it will be non-zero, and its actual magnitude, including typically big cache-footprint knock-on effects that show up attributed, in perf results, to mainline processing, will be practically impossible to estimate reliably without comparing against a rewrite.
So, instead, you generally have to say: we compared some similar(-ish) program Y that was rewritten and cut the number of server instances required to meet demand by 30%, 60%, or what-have-you. But, exactly for the reasons you cite, comparisons published are against performance under GC after that optimization has already been done, as much as was practical.
It sounds like you're most familiar with, and you're arguing from a situation where the cost/benefit tradeoff overwhelmingly makes efficiency the king. So sure, in that situation, don't use GC. Write code with those cost/benefit tradeoffs in mind. That's valid, probably awesome and wonderful in certain situations. However, that doesn't mean that those particular cost/benefit tradeoffs rule over everything, everywhere, and everyone else should follow suit.
that optimization has already been done, as much as was practical
Which could mean a lot of things. I think, for the sake of your argument, you're arguing that we can assume it means, "very close to maximum optimization." Sorry, but you can't make this assumption everywhere and always. It depends. As Matt Easton says: Context!
Indeed, a very great deal of code running on servers is in Python or Ruby. That's the low-hanging fruit before you start talking about GC overhead in a compiled language. But of course most of that is somebody else's code. People tend to be interested in what it costs to run their own code, not the next office over's.
If you believe your own argument, it would be foolish to start a rewrite while easy GC optimizations still await. Not everyone chooses wisely every time, but it's the charitable assumption.
Aside from that, rewriting something in C++ based on code in a higher-level language with better abstractions might cost additional developer time, maintainability, and quality.
But they're already advocating for a rewrite, just within the same language.
The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.
I think using term "are abstracted away" is better choice here. You still allocate memory, dosen't matter if it's malloc/free, mmap/unmap or compiler is doing it for you. It cost time and space. Sometimes the cost is negligible but not always, depends on application.
I don’t think that was their point.
> The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.
This is at least slightly misleading, though.
Obviously, memory comes from somewhere. You can amortize costs by not needing to allocate new pages often.
Minimizing the amount of memory consumed by an application dynamically is the only way to absolutely reduce the cost. There’s lots of ways to do this, and plenty of C++ and Go software aim for “zero allocations.” However, there is still actually allocations in many softwares with “zero allocations” because they still use the stack.
For deeply concurrent applications, the stack ends up being a lot of memory. If you reduce the amount of stack memory per fiber, it reduces memory usage initially, but then fibers are more likely to hit the guard page and allocate more stack.
There’s strategies to reduce dynamic allocations pretty much all over (even in GC’d languages like Go.) The fact is, though, avoiding it is much akin to avoiding the GC. In Go, its actually identical to avoiding the GC.
(As a note in post, I acknowledge that not every concurrent application uses fiber style concurrency, but I believe with minor adaptations this point still stands for many classes of applications. Fully avoiding OS allocations is possible, but it definitely isn’t the “default” for C++ apps.)
——
This isn’t to say your point is not correct at least for some viewpoint, but it’s not actually that simple, which is absolutely worth noting.
In that case you don't use std::string, std::vector... or any containers at all? Or anything else that allocates, directly or indirectly.
And, the answer is that real programs in obligate-GC languages spend overwhelmingly more time in GC than C++ or Rust. Much of this time is spent waiting on cache misses, which are hard to track to the responsible bit of code.
You're right that the GC isn't necessarily the issue. It's more the forced heap allocation which most GC languages come with.
That's actually true, as long as all things are equal. GC can amortize memory management cost. Of course at cost of more jitter.
Unfortunately many GC language users tend to do way more allocations as well, diminishing the advantage and even turning it into negative.
That's not really true anymore after the introduction of smart pointers. Basically C++ implements reference counting to manage memory as a result.
But we know that GC always imposes huge costs that point benchmarks uniformly fail to reveal. Often the costs are tolerable, or even negligible. At issue is the set of recourses available when they turn out not to be.
In fact, it can be worse in Java because you have no control over whether an object lives on the stack instead of the heap.
Make no mistake, rewriting is a huge expense, rarely embarked upon without readily demonstrated benefit. (Exceptions tend to be rewrites in Java for organizational / political reasons. But I digress.) Not spending the time, instead, adding features, and delaying new features until there is a place to put them, can dwarf the base cost of the rewrite. That rewrites are done frequently enough to be discussed tells you there are huge operational benefits available.