Hacker News Reader

Comment by zer0x4d | Hacker News Reader

Incredibly happy to see this series of CF articles. I was always so proud of devs back in the days where RAM and processing were scarce and who had to get creative to fit even the most basic stuff in the budget. It seemed to me that after RAM and processing became abundant, most gave up on optimization and focused on shipping instead which meant now that even with several cores, a basic notepad or music player failed to work. In a way, RAM becoming more expensive has ushered in a new era of forced optimizations, which I'm really happy for

View on HN

I don't remember those days with a ton of fondness. Yes, the challenge was fun, but I really wanted to ship it and get my product in the hands of customers. Now I can spend more time thinking about what they want and less time about what the computer wants.

Yes, I remember those too. The costs of manual memory management were real and were not low.

But costs on the cloud are real too, especially now. I’ve been living in JVM land for a very long time, but now it’s especially clear how important lean services are. Especially now that the bar for writing lean code is so much lower: let the borrow checker figure it out, etc.

I just spent a couple days wringing out more performance/memory efficiency for our services. Nice gains to be sure, but it’s still so immensely wasteful compared to something well written running native. If it was my money, I’d be going native for sure.

> But costs on the cloud are real too, especially now. I’ve been living in JVM land for a very long time, but now it’s especially clear how important lean services are. Especially now that the bar for writing lean code is so much lower: let the borrow checker figure it out, etc.

I don't think even Cloudflare bothers with this waste of time. If they did, they would certainly not have built their global infrastructure on JavaScript running on V8. They'd have done what Google and old-time Facebook did and built their whole infrastructure on low-level system languages, and hiring the world's leading minds on the subject to milk the last drop of performance from their hardware.

Even Google stopped to look at the problem and came up with Go. Not V8.

Google came up with Go because:

> The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.

From Rob Pike

All those things happened before the era of decent AI.

Also, it really does matter what you're building. Yet another CRUD app? Don't waste time on (super) lean languages - use Go or something else you like and move on.

Building a Kafka replacement? Making something that processes gobs of data quickly? Perhaps think about using something lean and efficient, as the economics have changed. "waste of time" (or memory) also applies to the cloud and your (or your company's) bill. And if you haven't noticed, you're being ripped off on the cloud, running something 10-50x less efficient has real bottom line impact.

Different people are different.

Some of us find production and optimization more interesting than marketing and distribution.

And its this obsession with shipping things as fast as possible quality be dammed that got us basic weather apps that eat a gigabyte+ of RAM.

And yet the world still turned

The world turned millenia ago and will turn millenia later.

The problem statement is of applications using up expensive RAM. Incidentally, expensive RAM is just one of the problems we face in the computing space. Forced obsolescence is another, when running hardware needs to be replaced because software is built for only newer CPUs.

> because software is built for only newer CPUs.

You are right, but it reads (to me, could be just me) you mean newer specs which used to be true when I shipped software in the 70-90s; you mean faster machines / more memory right? Like installing a new version of software or OS and suddenly all memory is used, system is swapping and you did not ask for that but some obscure feature you didn't need needed to be shipped fast.

> The problem statement is of applications using up expensive RAM. Incidentally, expensive RAM is just one of the problems we face in the computing space.

You need to stop and think about the problem. Nowadays RAM is expensive because many people now want to max out their computers with RAM to run LLMs and AI coding assistants. Today's software is still the same software that ran perfectly well half a dozen years ago. Cloudflare happens to operate a large global computer infrastructure, and it's scale is such that 1% gains are lauded as fantastic cost savers. But that's the bean counter's perspective, pointing out that they saved a bean.

> Today's software is still the same software that ran perfectly well half a dozen years ago.

This statement is quite honestly not true at all. Just take the two largest OS from 6 years ago and compare resource usage between them and you will find you are incorrect, never mind the software running on it.

> Nowadays RAM is expensive because many people now want to max out their computers with RAM to run LLMs and AI coding assistants.

Source? I imagine the amount of people trying to run local LLMs is miniscule. RAM is expensive because a handful of companies have spent billions buying all of the compute.

Sam and Dario really maxed out their rigs.

Thankfully the world's turning is not decided by big tech

Probably not for the lack of trying.

That statement can be used to justify anything, to the point it is utterly useless.

— Humanity has been on the decline. People are hateful towards each other, striking their fellow man and poisoning the environment. Despots eventually launched nukes which killed everyone but the cockroaches.

— And yet the world still turned.

For the rich, those in developing countries often were behind

On the contrary. Industrial developing countries saw bigger % increases in their economic growth in the last couple of decades than the fully developed "old" western ones. Mostly due to "the rich" shipping western jobs there to cut costs and increase their margins.

For example the likes of Poland, China, India and Vietnam were and still are growing like crazy when you compare to the likes of UK or Germany.

> And its this obsession with shipping things as fast as possible quality be dammed (...)

You seem confused. Allocating more memory than optimal levels is not a measure of quality. Similarly, a web page is not suddenly lower quality if an image asset is 50kb instead of 25kb. And how much complexity and engineering effort and bugs are you willing to tolerate to halve your memory allocations?

You are conflating quality with mindless minimization, not even knowing or caring that are the tradeoffs. The blog post you're commenting on starts by presenting the case for celebrating small improvements, even 1% improvements at a time. A similar 1% improvement in a mobile app is at like 1MB. Do you ever notice it? How many hours of engineering effort are you hoping to spend on this nonsense? And you prefer to spend it on this or in actually fixing a bug or implementing a feature?

This puerile conflation of minimization with quality suggests your personal notion of quality has no bearing on what quality actually is.

> And how much complexity and engineering effort and bugs are you willing to tolerate to halve your memory allocations?

You seem to be confused about this relationship. It's usually exactly the opposite.

The wasteful applications are generally not well reasoned about and half assed implementations. That's why they're guzzling resources

There is ofc a middle ground, because targeting eg incredibly resource constrained embedded systems will naturally increase complexity, but that's something entirely different to the scenario this discussion was about up to this point.

Exactly, that`s why i refuse to call 99.9% of "software engineers" engineers. Imagine you optimize a aircraft turbine 1%....man you get to drink a pool of champagne with the highest ceo's and aircraft carriers try to buy that turbine as fast as possible.

But with software and a install base of some millions 1% optimization is seen as wasteful, its like software slop is acceptable since forever because hardware gets faster, and electricity is "green" anyway.

Resource constraints have historically been a bastion of quality engineering, though. The relationship is not directly causal, but any time an engineer must not only solve a problem but _fit it within some kind of a budget_ vs having an unlimited budget, it forces them to slow down and think more carefully about what they are producing and how well it actually functions.

Ultimately it is an expression of anti-fragility: some minimal challenge must be met and conquered on many different axes of a production in order to help the full production itself mature to the best possible quality.

It is the same reason we exercise, the same reason that cars and toilets and so many products greatly increase in quality after emissions and usage and waste-related regulations get applied from above, etc.

The appreciation of the refining force of outside limitations is not about min-maxing, but it is at least partly about curbing the min-maxing of other concerns such as "ship the fastest garbage possible to move on to the next opportunity to repeat that process".

incredible way to put it!!

(Cat reading newspaper) I should build a database out of pointers

I wish I could care more about what the computer wants (because it’s fun). I couldn’t care less what the product demands.

"Now I can spend more time thinking about what they want and less time about what the computer wants."

What if they want memory efficiency

They wanted that in the 90s, didn't work out well as a software product https://en.wikipedia.org/wiki/SoftRAM

To rephrase the question, what if they wanted software that used less memory, i.e., software that used memory more efficiently

The OP, GGP and GP are about reducing memory usage by software

Pretending the parent question is about a different subject is unconvincing. The context does not support it

The vast majority of people do not care about memory efficiency on the levels we're talking about. I've never met a non-technical person who has cared. And out of the technical people that I know (which represent less than 5% of the population), no more than half of them have complained about the memory usage of an electron app.

So this hypothetical is wrong.

Or, they care but they lack the vocabulary to express what they care about.

You can get a new laptop to run software A or competing software B.

You can pay $1000 for the laptop with 4GB of RAM, and software A either can't run on that or will act like pulling teeth, so that software requires you to upgrade to 16GB of RAM which will put you back another $2000.

Or you can run software B which will hum along perfectly smoothly on the original $1000 laptop.

(All of that of course simplifying right past "everyone runs every app 24/7 with five trillion browser tabs open")

Are you honestly suggesting that the end user doesn't care about having to throw exponentially more money at stuffing RAM into their PC just to make the game all their friends are playing online function?

> Or, they care but they lack the vocabulary to express what they care about.

That's speculative to the point of being conspiratorial.

> Are you honestly suggesting that the end user doesn't care about having to throw exponentially more money at stuffing RAM into their PC just to make the game all their friends are playing online function?

I'm extremely clearly not suggesting that. Don't make up things and pretend that other people said them. Engage honestly or not at all.

Customers often want to pay less, and shareholders often want lower capital costs. A tasteful optimization is a win-win. The opportunity cost should be traded off against new features of course, but tasteful optimization is a good thing for customers.

> In a way, RAM becoming more expensive has ushered in a new era of forced optimizations, which I'm really happy for

Isn't this mainly Cloudflare's scale though? That's literally also what's in the introduction written as the reason why they are doing the optimization

While it's nice to see this focus, while they highlight absolute figures, we are still talking about 1% improvement. For most other software systems, nothing worth putting effort in for.

During the Moore's Law years you didn't have to optimize much, if you did a significant release yearly computer hardware grew faster than your optimization problems.

Reply on news.ycombinator.com