In Defense of RAM
404media.co
404media.co
I get the trend to soldered-on memory. I don't like it from a longevity standpoint, but it does provide space and performance benefits. But why are all laptops so low on memory?
If you look at annotated die shots of these CPUs you can see that the memory controllers use significant space on the die. So if you wanted to add more memory you needed to remove something else on the chip. Also you might need to change the cache hierarchy and the layout of the chip.
Like everything else in chip design this is a trade off. And the chip designers make different trade offs for server CPUs that can support up to multiple terabytes of RAM. If there were a big enough demand for a mobile chip with more RAM they would probably make one.
[1] die shots: https://twitter.com/highyieldYT/status/1719379808516280716
Edit: I might have misunderstood the OP.
The two biggest reasons are laptops have only 8 or 16 GB memory because most people don't need any more than that and the people that need more are willing to pay the markup.
So it's hard to chock this up to anything but nonsense.
Another reason is that the high-end laptops are a low volume product so this increases the cost of handling them. So naturally you would have consider this in the price.
But if you're interested in why the Thunderbolt cable costs more than charging cables, some folks put one through a CT scanner recently:
Let's take another example from Apple:
I hear they roll very smoothly, and since they don't have brakes you may need to re-level your floor to keep the computer from rolling away
Most customers don't purchase laptops by RAM capacity and instead by the cheapest crap with a bright low resolution display. It's too tempting for companies to remove the second memory channel and half the memory capacity to reduce cost. Why would they loose money on long lasting fast machines if the cheaper to crap sells a higher larger margin (and has to be replaced frequently)?
That's very much the point of soldered-down RAM, isn't it? 64GB kit (2x32GB) of 260-pin (SO-DIMM) DDR4 modules show up around $110 on amazon. But why would manufacturer give customers such option, if they can be forced to buy whole new laptop instead?
https://www.anandtech.com/show/21069/modular-lpddr-becomes-a...
On desktop the impact of power consumption is lower, so it is better to offer a slightly higher spec base model.
The Lenovo P-series workstations offer 128GB for the low price of $669
Penny pinching, artificial segmentation, and planned obsolescence.
I don't think upgradable RAM is coming back to Apple Silicon Macs. Upgradable storage, maybe; and there might be a use for standard RAM as a faster Swap, e.g., in a Mac Pro, to have 256 GB of super fast unified RAM and additional TBs of DIMMs for workloads that need that.
Nitpick: Something is off with the timeline of M2 Pro Macbook Pro ownership mentioned in the article ("Two years ago, I got a 14” MacBook Pro with an M2 Pro processor and 16 GB of RAM.") M2 Pro Macbook Pro were only introduced in January 2023.
Apple at a minimum should create a cadence of RAM upgrades to coincide with processor upgrades to at least some extent. Otherwise they are selling base model computers that are going to require most users to replace sooner. Conflicts with Apple's environmental efforts.
I think this is an interesting point, because my own experience completely disagrees. I feel like consumer laptop RAM has been stagnated at 8gb or 16gb for about decade now, which is insane as someone who grew up during the turn of the millennium when it was doubling yearly.
Among myself and people I know, the biggest factors for getting a new machine are usually (in order) battery life/consumption, physical condition, storage, and OS support for the hardware. I haven't heard someone say they need a computer with more RAM unless they started using it for new purposes they didn't before (someone getting into video editing, data analysis, etc).
The whole implied argument about hardware longevity is because software bloats. A responsible thing would be optimizing software for power and resource usage.
If it fails in your macbook, or any brand where it's soldered on, you're left with an expensive brick.
I believe that being able to easily and cheaply replace those components matters.
In regards to upgradeable RAM I'd rather not see it in the laptops. Too much power and efficiency left on the table for something with an overall audience who will rarely utilize the ability to swap it anyways. I do like the swapability in things targeted at more niche audiences though. I think Intel's approach with on chip memory in some of the new Xeons is interesting, you can use it as RAM or as a cache for your actual RAM should you need more than can be put on chip. Could fit well with the Studio/Pro line audience.
Thing is, HBM is a questionable fit for laptops because its extremely expensive, and pretty high power.
What Apple does is essentially what smartphones have been using for awhile, just scaled up. Its a good idea, and AMD/Intel can do it or even improve on it if they want.
The issue is that HBM needs like 4096 pins per connection or something absurd like that. So the only way to run HBM is through die-stacking (where the 4096 pins all lineup and the chip-manufacturer solders them together in a cleanroom).
HBM uses incredibly low amounts of power as a protocol. But then it jacks up the pin-count and runs thousands of the protocol in parallel to increase effective bandwidths.
In any case, you won't be able to physically route HBM on a PCB. The trace count alone makes it impossible.
I'd assume HBM is closer to DDR5 or GDDR6 than LPDDR5, but honestly I don't know. In any case, I don't personally consider DDR5 to be very memory-saving.
------
LPDDR5 on Apple Chips has a substantial power advantage. But like HBM, LPDDR5 cannot be socketed or expanded like DDR5 can be. LPDDR5 gets its assurances from either stacking on top of the die (like in Apple products), or by directly-soldering to the motherboard.
--------
The technical discussion here is about the socket. Should computers have a socket for RAM? Sockets require more power due to the lower-quality connection, but they allow for far more expansion and customization of the overall computer.
https://www.anandtech.com/show/21069/modular-lpddr-becomes-a...
Apple products have chosen to have higher-speed RAM that has lower capacity. PC users have the more standard (higher-capacity, but slower) RAM modules.
I'm sure everyone remembers the crazy MiniZinc benchmarks on Apple products, as well as other incredible RAM benchmarks. Yeah, when you directly solder RAM to the motherboard, and/or directly stack the RAM on top of the CPU, you can make more assumptions about communication... and that channel can likely have lower capacitances or other physical features.
Even 10-centimeters is about 2-clock ticks (assuming 4GHz clock), aka 0.5 nanoseconds, worth of distance. So that physical distance is in fact a substantial distance away when we're talking about nanosecond-scale communications that regularly take place between a CPU and RAM.
---------------
But its a tradeoff. By having the assumption that RAM is further away in typical PCs, we can stuff more RAM 10-centimeters away or 20-cm away on larger server motherboards (aka: sticks-and-sticks of different RAM).
But Apple makes a much smaller assumption: 0-cm... directly soldered on top of these M1 chips. Guess what? After 8GB of RAM, they've run out of room and they literally can't expand anymore unless they increase the assumed centimeters worth of distance out.
BTW: Thank your EE for equalizing all the trace-lengths to all your RAM chips. Don't believe me? Take a good look at a motherboard close to the RAM and/or CPU, I guarantee you'll see the "wavy" lines of PCB-traces as the EE in charge of the PCB-design is trying to trace-length match all the different wires to the RAM.
Author mentioned this in passing, but I think could have reflected more on how it is so much ram is used to do so little, lest they find their 64 GB machine obsolete as well.
(I think $1800 is an absurd price for an 8GB laptop.)
Embedded RAM, like or not, is the future of CPUs unless you want memory performance/power efficiency to freeze near current levels. Doubly so for graphics heavy SoCs like the M series.
But Apple motherboards should absolutely be swappable, Framework style. This would not be a huge technical compromise like swappable RAM, or a design compromise like a socketed M CPU.
And maybe we can get CXL "overflow" memory slots in laptops where performance is slow, but still 100x better than swap on SSDs.
Why is that?
This is already a serious problem.
For places where you are willing to give up power efficiency, or latency, or both, this is not as big a deal - this is why you see CXL/etc.
But for a laptop, where people want power efficiency, latency, etc, this is a serious issue.
SODIMMs are already "past" the limit, requiring 1.35V for modest DDR5 speeds at terrible timings, hence vendors are trying to migrate to this:
https://www.anandtech.com/show/21069/modular-lpddr-becomes-a...
I suspect regular desktop/server DIMMs are getting close to their limit as well
Essentially, the closer the RAM sits to the CPU, the faster/lower latency/lower power it gets. This is especially important in graphics heavy chips, where the wide, high frequency busses can eat loads of power. In fact, I wouldn't be surprised if Apple transitions to something "new" like a newer generation of Samsung's Wide I/O if they can get memory vendors to produce it in bulk.
The AMD Epyc has 12 channels of DDR5-4800 and manages 460GB/sec. It's a physically large chip, can handle a ton of ram, but also uses a ton of power. You can fit a crippled version (without all the memory channels) on a micro atx motherboard that's several inches thick. Normally you'd need an ATX, EATX, or rack mount motherboard. It needs several 100 watts for just the socket, not to mention the numerous dimms (12 minimum for 12 memory channels). It's the kind of thing you see in a data center, after you put ear protection on.
If you compare the M3 max, which fits in a laptop or small desktop, to Intel and AMD competition that fits in a thin laptop you end up with 1/4th the memory bandwidth. The competition either matches the power use and is much slower, or is as fast or faster at a huge power penalty. AMD and Intel laptops need discrete GPUs to compete and often require wall power to run at full speed.
All in all the M3 max is an impressive chip, fast, and very efficient. But part of that is the soldered in dimms that allow for great bandwidth and low power use.
A better AMD comparison is Van Gogh (the Steam Deck chip) to the base M1, and on paper they are very similar. The M1 is just a more "premium" version of the budget AMD chip.
AMD could make a good M3 competitor (and maybe even improve on it by tiling the memory controller/cache like the 7900), but... There is just no market for it. Windows OEMs dont want a big expensive SoC, they want a cheap CPU to hang a power hungry graphics card off of. Hence the successor to Van Gogh was canceled because no one wanted it.
AMD even makes better memory systems for the XboxX and PS5, which run games great. For years gamers drifted from gaming desktops to consoles because GPUs were insanely priced or unavailable.
However, finally, many years after the GPU shortage there's GASP going to be a laptop/desktop chip with iGPU with more than a 128 bit wide memory interface. The AMD Strix Halo. I hope it does well, it's roughly similar to the M2 Pro. Still half a M2 Max or 1/4th a M2 ultra.
Eh, I was excited at first, but rumors suggest its going to be a tiled, power hungry SoC with high clocked cores. Its like the desktop-class 7000 series monsters they stuff into laptops, but with a decent IGP hanging off... All performance and zero efficiency.
I have higher hopes for Arrow Lake: https://videocardz.com/newz/intel-arrow-lake-p-with-320eu-gp...
Hawk Point, Strix Point, Strix Halo, and Dragon Range look to have a decently wide range of performance and power points coveraged. The top end will have the X3D (extra cache) and the next tier down will have double the memory bandwidth.
Or maybe pigs will fly and Apple will lend a hand to Asahi to get Linux working well. Not that Asahi hasn't been doing a good job, I do wonder how long the new GPU features will take to show up in Linux.
Ryzen dual channel bandwidth is 100GBps. EPYC 8 channel bandwidth is well under 400GBps.
* No M2 memory benchmark I've seen gets above ~250GB/s once cache effects are exceeded.
* Random access latency is >~110ns which is no improvement over even mid-range DDR4 designs from 5 years ago.
TL;DR: It's about cost
It's horrible because there's actually no "fix" I can offer these normal people who made the mistake of buying an unupgradable 8GB computer.
Technical people understand the limitations, and work within them, and aren't disappointed when they know they're hitting the upper limit. Normies want a cheap laptop, but don't understand why their bottom-tier Apple laptop can't run everything at once.
But most users are probably disappointed and expected more from it, so I understand why they weren't able to continue that line
I would describe it as "no unexpected problems that aren't natural consequences of choices": Apple's memory and swap management is actually shockingly good under the circumstances, it does thrash the SSD a bunch, and it locks up hard if put to sleep under heavy memory pressure. One mitigation is just leaving it awake overnight before any critical demos, etc.
EDIT: I was wrong; it is a 16GB model, 32GB was the ask, performance is as described, it feels smaller; 8GB would probably be totally unusable.
The 16" M1 Pro came with a minimum 16GB of RAM.
https://support.apple.com/kb/SP858?locale=en_US
So did the 14".
0: https://en.m.wikipedia.org/wiki/MacBook_Pro_(Apple_silicon)
I have an M1 Macbook Air with 8GB RAM. The choice between 8 and 16GB was strictly a monetary one. I wish I'd bought 16GB, but I don't regret buying 8GB. I mostly run native apps, close down ones I'm not using, a rarely have more than a dozen Safari windows open at once. The only Electron app I run is Obsidian. I'm using some swap, but have never had an out of memory alert in three years. I hope when it comes time to replace my Air, I'll be able to afford more memory. Until then I'm doing just fine.
But you could upgrade it yourself, of course, and that’s what all the old Macheads would advise you do.
These days… urg! Their SSDs are ridiculously tiny too, which I think is even more egregious. My computing habits might not require more RAM in the future, but my storage requirements always go up.
The real key here is the PCIe NVMe bandwidth. If macos could swap in and out all of two entire memory heavy applications in a fraction of a second, the RAM size isn't as important as it used to be.
1. How macOS uses RAM changed a few years back. Memory Pressure is just important of a metric as usage because data is intelligent cached to fill memory. The author’s screenshot shows there’s no memory pressure on their machine while arguing there’s not enough memory. My RAM is usually pretty “full” but a lot of it is pre-loaded data.
2. It seems like the author uses quite a few Electron apps. Maybe I’m shifting the blame here but Electron is a known resource hog. I’ve seen Signal use more resources than a full Linux VM. Native apps are usually faster, less resource intensive, and oftentimes follow the HIG / have better UX. For the user there’s no downside.
3. There’s different “pros” out there. So often us developers think we need these big dank machines just to work inside Vim all day. I’m extremely guilty of that. And having headroom is nice. But even an entry MacBook Air could do the job. It’s not as fun but it’d be fine.
It’s fascinating how we all try to extrapolate assuming our usage is the norm. I do the same. But we all have different expectations.
Seems like it should easily last 5 years even with constant full utilization of 8GB RAM.
I thought I had read when the M1 came out that they had memory compression turned on by default but I'm not an Apple user.