Hacker News Reader

Comment by PeterisP | Hacker News Reader

It's amazing what performance you can get by simply putting in as much RAM as your data needs - if plugging in a terabyte or two of RAM ensures that all your data is "in cache" on a single system (+ e.g. a mirror for redundancy) instead of having to constantly read from disk or do horizontal scaling, that's a relatively cheap way to get a simple&performant system.

View on HN

If it’s easy/cheap to build 1-2 TB RAM systems, why are there so few I can rent from AWS?

I settled for 768GB when I tried a few months ago and it wasn’t cheap. They did have like one HANA class system that was multi-TB at even crazier prices but I was kind of disappointed when I went to do something like you suggest. It was also impossible to get a lot of RAM without overpaying for a lot of cores I didn’t need and couldn’t use.

I don’t disagree with your point about the optimization benefits of just caching data but actually getting a cache optimized system is seemingly another thing entirely. My perception is that it’s been hard to get >768 GB systems for a bunch of years now. But I don’t follow this space closely,

Actually, this is one of the easiest strategies to write slow software, because you're relying too much on the hardware to bail your poor decisions out. When running the software on worse hardware, you will see super linear slowdown.

This is from the perspective of designing a custom system for doing a particular business function, and its software and hardware are interlinked parts of the whole design (as would be e.g. a cloud platform if that was used for it), and it makes all sense of always considering them together as that enables valuable tradeoffs of one vs another instead of assuming that "this hardware must be powerful&efficient for any system" and "this software should run on any hardware". The software isn't intended to ever run on any other hardware, so in that context questions like "running the software on worse hardware" are meaningless as the software is not an isolated component meant to be deployed somewhere else on unknown hardware, it runs exactly on the hardware you want, and if you can save $5k of software costs by spending $1k of hardware costs then you should do it just as eagerly as if you could save $5k of hardware costs by spending $1k of software work. (Which is quite different from mass market software, where you have to consider small runtime savings multiplied by vary many different consumer devices running the same software)

But for many companies building such systems (custom line-of-business software is probably still the largest part of software development happening in the world) it's highly probable that if you can "bail your poor decisions out" by just buying RAM, that's is far more efficient than rebuilding parts of software - a saved man-month of engineering can buy a LOT of RAM; a terabyte of it is probably in the range where it's not even particularly worth trying to carefully evaluate the options as a well-staffed meeting or two discussing the possibilities can be more expensive than buying a few RAM sticks more.

This sounds straightforward but you're glossing over a lot of second order effects. Notably, if this idea dominates your attitude you'll be much more likely to miss the cases where a bit of judicious optimization actually would save substantially more in hardware costs. And it may not be 1k of hardware costs, but hundreds of k per year of cloud compute costs, depending on the situation.

I've been getting my hands dirty root causing a latency issue in another team's codebase at work. The program needs to run in real time, but the language is JIT compiled and garbage collected. In my most recent experiment, I successfully eliminated the most egregious spikes by completely turning off garbage collection during the latency sensitive phase. RAM usage creeped up to 50GB, but my workstation has 64GB so good enough? Also, I need to run the program 3-4 times to cache all the code paths, given that the program behavior is latency dependent. Just to make things extra spicy, the program seems to also spin off a rogue process that fills up my 1TB SSD and then die after a few hours.

One of the first people I know that "worked in computers" when I was a kid had a really fast 386 running Windows 3.0

I complained about how slow my 1 Meg 386 was and wanted a 486, he said he had 16 megs of ram (kind of insane at the time) and always said "the system will wait on you"

That was somewhat true until you went to load a new file or program that wasn't cached yet, or when a heavy re-draw occurred and a 4mb 486 would dust it. You just didn't page anymore, which happened a lot back then.

Reply on news.ycombinator.com