Hacker News Reader

Comment by lockhouse | Hacker News Reader

Processing power is fine these days. It’s memory that I feel has stagnated.

The standard computer configuration has been stuck at 8 GB of RAM and 256 GB of SSD storage forever.

> Processing power is fine these days.

I don't know, I've been working with LLMs a lot recently and for the first time in a while I am wishing I had access to much more compute than I do. Imagine having the power of a H100 locally without having to pay thousands of dollars a month.

For inference, at least locally, the bottleneck is usually the memory bandwidth (and quantity, of course).

I hope that AI hype lead us to more memory and more memory bandwidth, because they are really lagging behind computer power increase from like 15 years already.

Oh, 100%. But you can do some pretty amazing things with fine-tuning LLMs too, and that is very compute intensive. Not to mention it's ridiculously hard even getting access to a cloud GPU instance nowadays.

Get used to it. The memory wall is coming and if you are in the industry it's possible that within your career you may need to adapt to falling DRAM-to-core ratios.

The memory wall arrived decades ago. That's why caches were invented (which don't solve the problem, just soften the blow).

One approach would be to snip RAM in as many pieces as you have cores, and attach each piece locally to a CPU core. Say eg. 1k cores each with 1/1000th of total RAM, accessible at L1 cache like speeds. Giving you crazy-high (combined) memory bandwidth. I know, there exist some IC's that actually do this.

Problem is such a setup is not suitable for all types of computation. For data structures that require individual cores to access other cores' local RAM, you still need some communication protocol (+ latency, bringing you back to square 1).

And we haven't quite figured out how to program such a beast in software developer-friendly way.

Hence the usual approach of "all RAM in one pile, connected via a fat pipe to a heap of CPU cores".

You're talking about time but I am talking about space.

I'm not sure what "standard computer configuration" means. Maybe you mean a budget laptop? Your typical new gaming desktop build is 32GB, and for a workstation probably 64GB.

I think you can get a 2TB ssd for like a 100 bucks nowadays. They are dirt cheap.

Gaming desktops are not even close to being a standard computer configuration and you know it.

I’m talking about the typical computer you’re going to find in a big box store or the default configuration on a website.

Yes you can add an SSD yourself for that price but the markup for configuring it at purchase is considerably more than $100. I’m not talking just Apple when it comes to this, Dell, Lenovo, HP, Microsoft, and just about everyone else do it too. RAM is even worse now that many (most?) laptops have it soldered in, so there is no DIY upgrade after you buy.

> RAM is even worse now that many (most?) laptops have it soldered in, so there is no DIY upgrade after you buy.

From the cursory look to an e-shop, it seems to me that majority of notebooks have soldered-in memory, but also one SIDIMM slot for expansion. Just the very low-end ones (and Apple) have no SODIMM slot.

And 16 GB is definitely the most populous category of notebooks.

Look harder. LPDDR isn't for low-end laptops, it's for low-power laptops. All the major brands are using LPDDR in their thin and light systems.

Sorry I don't follow how this "standard" configuration is relevant. If you want more just buy more? If you don't want to pay pre-built markups build it yourself.

And the point about gaming setups is very relevant because you can get consumer hardware (aka cheap), instead of being forced into "workstation/server" which carries a huge markup.

You can get 128GB RAM and stupid amounts of ssd storage for cheap with consumer hardware.

The hardware hasn't stagnated at all. I think you're problem is simply with pre-built markups?

As a developer I care because that’s what my users have. As an employee I care because that’s what my employer buys for me to use at work. As the unofficial IT support person for my friends and family I care because that’s what most of them will buy and ultimately ask me to help with.

Also if you want a laptop, the markups for upgrading said memory and storage are actually pretty significant and may not be worthwhile. Upgrading laptops is quickly becoming difficult to impossible. Yes I know about Framework, but they are not ever going to be mainstream for a number of reasons.

>I think you can get a 2TB ssd for like a 100 bucks nowadays. They are dirt cheap.

Unless you're paying Apple for it :^). Hope you like 12x markup to go with your 1TB SSD :^)

Out of curiosity I looked at the store I get my laptops etc. from and grouped by RAM, the laptop category looks like this:

8GB - ~350

16GB - ~1060

32GB - ~550

I don't know about desktop PCs, but in laptops 8GB is not mainstream any more.

it's gotten better somewhat.

thou, 8gigs are overly common (and overrepresented in regard to your market analysis) as most consumers (and enterprises) tend to buy the low-end config unless it's absolutely necessary to spend more.

The problem is the difference in pricing between the 16 and the 32gigb one, the manufacturers are taking advantage of the sodering which is almost going to be the default in the upcoming days, and taking consumers for a ride.

Von Neuman's architecture has run out of steam. The fact that most transistors in a computer at any given moment are idle seems to be a huge waste. What if you could just have a computational fabric that lets you have one instruction per cell, and run whole programs in parallel?

FPGAs do that, but the "smart" routing fabric in them makes compiling code to them take hours or days.

If you eliminate the switching fabric on an FPGA, you are left with a grid of Look Up Tables (LUTS) each connected to their neighbors. The result is a Turing Complete computer that works exclusively in parallel.

At home (which is also my workplace) all my PCs are at 128GB. The server is 512GB. Laptops are 64GB. RAM has not been stagnated. Just buy what you need. To get it cheap for example for laptops I would buy smallest configuration (RAM and SSD wise) but with good CPU. I would then throw out old RAM and SSD and replace with the ones I buy separately. Way cheaper this way. PCs and servers are assembled from parts. Again I just order what I need and then let custom PC maker nearby assemble it for me.

build your own, it's not that expensive to 4x both those numbers

"The standard computer configuration" stopped referring to desktops long before 8GB DRAM + 256GB SSD went mainstream.

For general office computing or media consumption, yes, the 'standard' is indeed laptops, tablets, phones, consoles and TV's.

For locally playing with AI, those are a joke. A beefy desktop gpu, preferably with 24GB vram, a nice desktop high threaded cpu. 64GB of ram and as many TB of fast M.2 storage you can afford is the starting point.

Those OSS LLM models will eat your typical enthusiast gaming PC for breakfast.

Depends on your demographic, I guess. My 8-year-old desktop was built with 16GB and that was middle of the road even then. Only a 64GB SSD at the start, though. I didn't see laptops really take over from desktops for a few years after that.

Your story doesn't really add up. 8 years ago would have been 2015, at which point 64GB drives had been dropped from most consumer SSD product lines for at least a year, and laptop sales had been outnumbering desktop sales by a wide margin for many years. I don't think you realize how thoroughly unrepresentative your experiences and purchasing choices have been.

This has gotta be peak HN. "I built a desktop 8 years ago using these components which were pretty normal at the time." "No they weren't." There were like 10 different brands of 64GB 2.5" SATA SSDs to choose from at the time, in the most popular PC components shop in the city, so it wasn't just my little niche. Plenty of people had laptops, sure, but most daily-driver work machines were still desktops.

Maybe it was different where you live. That's OK.

The minute you said custom built from components you went off the rails of my argument. There is nothing standard or common about that. Very few people build their own computers from components, nor should they. It also ignores the fact that most people buy laptops.

I am talking about the memory and storage configuration of the "typical" computer that an average consumer will buy from a big box store or a website. This has been 8GB/256GB SSD for a long time now. That is my source of complaint. Growing up in the 80s and 90s there would be absolute leaps in memory and storage every time you would upgrade your computer. That is no longer the case. We are just now starting to see some 16GB/512GB configurations appear on consumer default configurations after what, almost a decade?

> There were like 10 different brands of 64GB 2.5" SATA SSDs to choose from at the time, in the most popular PC components shop in the city, so it wasn't just my little niche.

Your story gets even weirder! Sourcing components from the local brick and mortar store when building your own desktop usually negates most of the benefits of building your own desktop computer. It's not entirely surprising that they had an excess of outdated SSDs on display.

> Plenty of people had laptops, sure, but most daily-driver work machines were still desktops.

Are we suddenly confining the discussion to corporate-owned office PCs, or do you genuinely not understand that laptops have been outselling desktops for a very long time?

Reply on news.ycombinator.com