For people who still don't get how bad DIMMs are
twitter.com
twitter.com
Intel is married to the commodity server market and their laptop/mobile products have always been trickle-down, because they are a giant megacorp whose roadmaps are dictated by their biggest profit centers. Even if they designed this kind of system, they wouldn't be able to capture most of the profit from it like Apple can, since they don't sell memory or systems, they sell CPUs. After the mobile debacle, Microsoft pivoted to SaaS and is way too chickenshit to start designing hardware. Apple is the only company incentivized to produce a product like this for consumers. This is not going to suddenly become profitable for the commodity server market; and it doesn't have to; it's not a market Apple is particularly interested in.
What does similar mean? I'm not sure I agree.
The M1 is basically a classic big-iron architecture, miniaturized with chiplets. The top M1 Max config is effectively 8x channel/512-bit-wide, just like a current epyc. It's faster though because it's 5.2GT/s LPDDR5 memory (400GBps) vs 3.2GT/s DDR5, and it's much smaller because it appears as 4x stacks on package. (Next gen Genoa is up to 12x channels of 5.2GT/s DDR5 fwiw.) But it's still a conventional play, basically, with new packaging.
> Intel is married to the commodity server market and their laptop/mobile products have always been trickle-down, because they are a giant megacorp whose roadmaps are dictated by their biggest profit centers.
Generally, yes! Notably sometimes innovation is first delivered in mobile and "trickles up," but basically it's because mobile is a safer easier space to innovate/explore/learn in (as it's much less investment cost to make the much smaller/lighter chips there). Generally you're right about what leads the business.
> Even if they designed this kind of system, they wouldn't be able to capture most of the profit from it like Apple can, since they don't sell memory or systems, they sell CPUs.
It's definitely a little intimidating figuring out how to sell bundled on ram. The technical benefits are hard to ignore though. Total cost of ownership should improve for customers notably, via less intense/simpler motherboards, via low power consumption stemming from much more direct channels & stronger signal integrity to main memory. But you're right, I think there's a lot of unappealing & unwanted aspects to becoming a company that ships ram. It'll be hard to resist/ignore for long though.
AMD hasn't shown DRAM related interest yet, but their 5800X3D has 96MB stacked L3 v-cache on a separate chiplet. Their upcoming Genoa cpus will have ~1GB stacked L3 V-cache, again at absurd terabits-per-second speed, also via chiplets. AMD is pushing the memory heirarchy wider, is doing something like this M1-ening, but at a at higher-throughput lower-latency but smaller-capacity points, while leaving the conventional RAM channels also available. But they could, pretty easily I'd wager, snap back & deliver integrated DDR if they wanted, saw the need.
Now that there are groups like Universal Chiplet Interconnect Express, this tech is kind of semi-commodified, available for purchase, to a degree. Yes, the mainstream market may lag. But with the capabilities arriving, it's hard to imagine this tech staying out of the data-centers for too long. Once we see some hyperscalers or upstarts delivering chips with integrated DDR, I expect the appeal will be irresistible/unsuppressable. It'll become obvious/known that this is good tech that is table stakes for selling chips, like it or not.
I was thinking that about HMC and thinking that HBM was still Samsung / SK's thing. I see now everyone is making HBM. You'd build your system around HBM.
> Notably sometimes innovation is first delivered in mobile and "trickles up,"
Not at Intel :P but I agree in general.
Apple would never use HBM because consumer margins and they have been building SoC based on LPDDR for years. Hyperscalers already do their compute on "bundled ram" systems when they buy NV Tesla or AMD ROCm cards, both based on HBM. AMD is targeting the slow&wide "million VMs" market, and DDR is doing great in their niche.
I do agree we will start seeing weirder server boards sold with soldered DDR for cost and power reasons, probably by Intel or AMD. I think it will be an even more specific niche, sitting between off the shelf commodity servers and expensive appliances like NV DGX. Again Intel/AMD are in a bad spot to capture the fat margin here.
UCIE is classic giving away old stuff to extinguish competition. It's good for bespoke builders but meant to ward off actual competition from selling something similar and then eventually becoming competitive with IF or NVLink etc. Note Apple made their own fancy interconnect thing too for M1.
AMD hasn't really sold HBM as far as I've seen since ~2018. Counter-point, upcoming Intel data-center GPUs seem loaded with HBM. I tend to believe this use case will be more permissive of the the higher latency, wider width HBM, that deep number crunching is less affected. Nvidia has HBM on their contemporary Axx Tesla accelerators- again, a case where aggregate number crunching with some internal latency is ok, unlike general GPU and CPU work.
Right now the Mac Pro has 2TB memory limit. Apple's either going to have to commit considerable board space to DRAM, use sockets or just completely obliterate the max amount of RAM a M* series Mac Pro can have.
Yesterday there was an article on Intel's newest integrated silicon photonics, this time having some of the photonics on stacked chiplets on the chip[1]. It's a different sort of integration and micro-miniaturization, but feels very much the same. Not needing a big ole NIC, not needing to have extra, power sucking PCIe transcievers at each end, by moving capabilities onto the chip complex would be amazing. I'd love to see ultra-dense servers of the future where there are dozens of cpus on a sled, with basically just power & optical io feeding each chip. We've bounced off the "disaggregated rack" a couple times now, but it also feels like a major phase transition has been building for a while to.
human manipulable components are too big for the interconnect and density we want now... and those are already too small for easy cooling.
What I cant forgive is Apples insistence on forced planned obsolescence by not including memory slot for additional expandable second level ram. They even got a stupid and probably invalid patent for that exact thing https://www.techpowerup.com/277760/apple-patents-multi-level... . You could have 8GB of normal/fast ram + 32GB sodimm buffer instead of burning SSD down with paging.
2. basically the same
3. basically the same
The advantage to Apple's soldered RAM is that the motherboard is 10x smaller.
The base model aluminium Intel Mac Minis started out with 2GB RAM, then 4 GB and finally 8 GB - all over a span of 8 years. Many of these models would be useless today if you couldn't upgrade the RAM at all. In fact, the 2014 Mac Mini models with soldered RAM (LPDDR) received a very poor reception and Apple was forced to backtrack and go back to user-replaceable RAM with the next Mac Minis ( https://everymac.com/systems/apple/mac_mini/mac-mini-aluminu... ).
Many of the early adopters of Apple Silicon M1 devices noticed heavy usage of their SSD, which many attributed to heavy swapping due to the limited (non-upgradable) RAM in the M1 devices. Apple finally said it was a bug ( https://appleinsider.com/articles/21/06/04/apple-resolves-m1... ) and recommended upgrading macOS. But many of these early adopters are less than satisfied and still spend a lot of anxious time worrying whether their soldered SSD's life has reduced by a year or two because of the increased usage. Their concerns are real because with soldered SSDs, you obviously have to do a very, very costly repair through Apple if the soldered SSD chips conk out.
Imagine if you have to replace your whole CPU / SoC to upgrade just your RAM because the RAM is now integrated with the CPU / SoC!? (This is ignoring that even CPUs are now unnecessarily being soldered in many Apple and non-Apple desktops and not easily upgradeable ...).
At some point, the cost-benefit analysis of such soldered / integrated parts doesn't work out in favour of us consumers. And that's what we need to consider before blindly giving up ease of repair for some brief performance increase. (While I have specifically highlighted Apple because they are the only one currently using integrated RAM with their CPU / SoCs, my criticism is directed at the others too, including Dell, HP, Lenovo etc. who are following the same path of planned obsolescence by soldering components to make them less repairable and reduce their life. Unless we start supporting repairability - like with https://frame.work/ devices - we consumers are in a very slippery slope of losing our right to own and repair our devices).
How do you upgrade your L2 cache? You're not still using a 486, are you?
Did you protest the Pentium as well?
But you can just move your unsoldered M.2 SSD to the new board. And when properly supported by OEMs, motherboards can last you for generations of CPUs.
There are B350 and X370 boards that were released with the Ryzen 7 1800X in 2017, that support the recently released 5800X3D, a processor two generations of microarchitecture newer on an entirely different node. And while that processor still has the same amount of L2, you get six times the L3 cache of it's predecessor.
No, it's not. Well, at least it wasn't intended to be.
The whole point of my post is: What were you supposed to do with all of the expensive cache chips you bought for your 486 when the Pentium came out?
--> They weren't soldered on, but you couldn't use them anymore if you wanted to upgrade your Pentium. <--
How is the situation with RAM today any different?
The only difference I see is that we now expect cache to be soldered on. I imagine in the future we'll expect the same of RAM too.
Yes, we expect cache to be built-in because the manufacturers of high-speed memory realistically don't expect the average consumers to be their target market, and thus chose the B2B model to sell their product. Nobody (that I know of) has also explored offering upgradeable cache memory as an option to consumers (hard to know if it even possible with current CPU / SoC hardware).
As for consumers changing their expectation to accepting soldered RAM and SSDs, that remains to be seen - anything is possible with powerful lobbying and PR. Personally, I think Intel and AMD would be fools to not treat any "integrated" RAM in their future CPUs as "L4 / L5 cache memory" while still fully supporting DIMM DDR RAM as that would make their CPUs more powerful and more desirable than Apple Silicon SoCs. Note that unlike Apple, they don't sell a closed-box solution and have to support multiple OS platform. Thus, it would be more difficult for them to offer a "one size fits all" built-in RAM with their CPUs.
The 486.
> Yes, we expect cache to be built-in because the manufacturers of high-speed memory realistically don't expect the average consumers to be their target market
The 486.
> Nobody (that I know of) has also explored offering upgradeable cache memory as an option to consumers
The 486.
The signals move between the cpu and the ram at about one nanosecond per foot. A nanosecond is one full cycle of a 1GHz clock.
The characteristic impedance of the connection is set by the physical size of the pcb track and the dielectric constant of the pcb material.
As the signals move between the cpu and ram at nearly the speed of light if they hit an impedance discontinuity some of the energy is reflected back. That messes up the signal.
Making a connector with hundreds of pins that works reliably and does not cause a significant impedance discontinuity is hard.
The lower the impedance of the connection, the more power is required to drive data through it.
A bit error rate of one in a billion would corrupt data or cause a software crash daily so signal integrity is a hard problem.