Hacker News Reader

Comment by ksec | Hacker News Reader

>Combined with die stacking, which allows for up to 8 dies to be stacked as a single chip, then a 40 element LRDIMM can reach an effective memory capacity of 2TB. Or for the more humble unbuffered DIMM, this would mean we’ll eventually see DIMM capacities reach 128GB for your typical dual rank configuration.

So on 8 Channel 16 DIMM per socket you could fit a theoretical 32TB of memory. This is insane amount of memory and great for In-Memory Database. ( How is Intel Optane going to compete? )

This makes me wonder, what makes DRAM so expensive? It is still hovering at a median price or around $3/GB compared to NAND which is less than $0.1/GB.

View on HN

> This makes me wonder, what makes DRAM so expensive?

Greed does. DRAM makers were antitrust busted at least 7 times on my memory in Taiwan, Korea, and USA.

Greed to me is a funny word here. If illegal antitrust actions are occurring, well that’s obviously bad for the consumer and market, but you say greed as if DRAM makers owe something to the public? They’re a technology business who exist to make money by definition. It’s not a non-profit.

That hasn't been the history of the business. DRAM makers are state-sponsored strategic enterprises which do not act according to rational economic theories.

Exactly. Compare that to privately held corporations, which would never think to illegally keep prices up through collaboration, just because they can reasonably expect to get away with it. /s

So it’s either or? Both can’t be right? Geez. Straw man much?

well you can either be state-sponsored or not. if GP is bringing up state-sponsoring as a supposed source of irrationality then yeah I'm assuming that they presume privately held corporations would do better.

that being said, I do admit my post was sarcastic, but it also did mention what the problem here is (besides a much broader discussion about captialism) which is that they can rationally expect to get away with this. State-sponsored ones because their interests are the national interest (to an extent) and privately held ones because they're usually able to effectively capture their regulators.

If that's the only reason, then you'd think companies like Apple with dedicated fab timeshare could cut costs by printing their own RAM chips, rather than buying them. And It's not like DRAM contains any complex IP they'd need to license—if you can lay out a CPU, you can lay out RAM.

The IP is mostly in the production process, not the circuit itself. They need to get (relative to logic chips) very low failure rates to turn a profit, which requires very extensive tuning of the manufacturing process.

My understanding is that DRAM processes are sufficiently different from logic processes that they tend to get dedicated fabs. So then companies like Apple would have to buy fab timeshare from RAM manufacturers that have no incentive (as far as I can see) to let them produce DRAM for less than whatever the bulk pricing is.

Yes

What's the reason, then, that anyone with the capital to do so (again, e.g. Apple) doesn't just build a RAM fab, to both get cheap RAM for themselves, and join the cabal selling marked-up RAM to others?

Seems like RAM manufacturing is like building a casino: expensive to do, but basically a sure bet.

The problem is that you need to have $10+B to throw at the problem after you license patents from the incumbents, and even then you'll end up being behind them technologically by the time your fab is up and running, so your marginal costs won't be appreciably better than contracting with the existing manufacturers for their more advanced memory.

You could theoretically spend several times more than that to try to get ahead over the course of two or three generations, but for that kind of money you could just as easily secure some very preferential pricing from one or more of the incumbents, thereby ensuring that everyone else trying to put a lot of RAM into a PC has to pay more.

The most viable path to establishing a new leading-edge competitor in this space is for a government to throw lots of money at the problem, knowing that it'll be years at best before it produces anything competitive, but having the advantage of being able to more or less ignore IP issues and having a potential demand far higher than any one memory customer can produce on its own. China is doing this for the NAND market, too.

There would be a lot of candidate oligopoly markets to attack if Apple got into that business with its war chest. A risk with the DRAM plan is that they could succeed in lowering prices but the costs would be unpredictable and they would then take a very long time to break even, givn the new lower margins, and would have to become a semi house that makes chips for other customers (to get required volume to break even), taking away focus from their consumer product business.

Apple probably will in the future when it is no longer able to squeeze margins from other expensive parts. It invested in the screen manufacturing now has moved to producing it own processors looking at the next most expensive part it is a likely move. Ram chip and flash drives if you are large buyer you might look at producing your own. If they are able to use all the capacity of a fab they might look at doing so. Or like they did with sharp invest in a company that runs a fab for using majority of the production.

The processes to form NAND and DRAM are completely different. DRAM relies on creating non-leaking capacitors which are highly difficult to manufacture at such a small scale. NAND benefits from innovations in the CPU lithography space since it's essentially all transistor based. Why would you expect them to have the same price, unless you knew nothing about the technology? Also, there are plenty of distinct competitors in the DRAM space. Do you have a source suggesting Micron and Samsung are engaging in price fixing together?

"Plenty" of distinct being Samsung, Micron and Hynix.

They are not necessarily price fixing illegally. It's just that they all keep their production and capacity expansions closely in check to not ever let prices down.

That and in rare cases when prices are down due to unexpected decline in shipments they're all very swift to shift wafers to produce something else. Feel free to dig DRAMeXchange reports for for details.

Samsung is one of the worst companies. Among the BP, AT&T, Comcast, and Apple tier evils.

I'm not sure how they have survived bad press, but Samsung is not a good company.

> NAND benefits from innovations in the CPU lithography space since it's essentially all transistor based.

It's really not, especially in the 3D NAND flash era where only one manufacturer is still using a floating gate cell. It's so thoroughly not transistor based that the Chinese upstart's claim to fame is that they fabricate the transistors on an entirely different wafer from the memory cells, and glue them together later.

It's best to think of NAND, DRAM and logic as three separate categories that each require a very different mix of tools in the fab, especially on the back-end. (But you won't be finding EUV or quad-patterning in the front-end of a NAND fab, either.)

If the difficulty of DRAM is in creating capacitors at that size, why haven't we seen a shift toward SRAM (6T, for instance), which is purely transistor-based?

Sure, you have to sacrifice more transistors for the same capacity, but newer processes can fit more on the chip, right? I recall from computer architecture classes that the benefit of DRAM is the ability to use fewer transistors, but if transistors are cheap...

(I'm sure I'm missing something here. Power consumption / heat generation? I also never really understood why SRAM continues to be so expensive, when it seems like it would obviously benefit from smaller processes.)

On Intel's 10 nm process you can make SRAM with a density of about 20 megabit/mm². [1] Older processes are much worse (<5 megabit/mm²). [2] A current-gen DRAM package achieves about 170 megabit/mm² (but that's two dies, probably stacked). This article [3] cites 8 Gb on 77 mm² on a 21 nm process, giving 105 megabit/mm², and 148 megabit/mm² for the DDR5 version with a die size of 54 mm². The same article shows a Samsung part with around 200 megabit/mm² density.

So even if you were to manufacture SRAM on Intel's ultra-expensive 10 nm logic process, you'd need a massive amount of silicon for the same capacity.

[1] https://fuse.wikichip.org/wp-content/uploads/2017/12/isscc-2... [2] https://d3i71xaburhd42.cloudfront.net/f20203949a744276e338d6... [3] https://www.anandtech.com/show/13999/sk-hynix-details-its-dd...

Totally makes sense that you wouldn't get the same capacity from the same silicon, or even close, given that SRAM uses far more transistors per cell.

But if you have issues scaling DRAM, and different scaling limits on transistor count / SRAM, it makes sense (to me at least) to start considering SRAM as an option (e.g. for lower latency, faster speeds, higher bandwidth transfers, etc). Just because you can't achieve the same capacity today doesn't mean there's no merit to it -- HDDs vs SSDs from a decade ago feels like the obvious comparison.

Supposedly [1] TSMC's 5nm process yields 256Mb on a 5.376mm² die, at roughly ~50Mb/mm², which would translate to a 3.5Gb die of the same size as the SK Hynix chip. Sure, that's no 16Gb die, but you could easily make 32GB sticks (assuming that you could just combine these chips in the same way as in DDR4).

I guess there's also a barrier to entry in that you'd also either need new hardware to deal with "SRAM sticks", or some sort of compatibility layer (a controller that implements the DDRx signaling logic, perhaps).

[1] https://www.anandtech.com/show/15219/early-tsmc-5nm-test-chi...

Forgive my naivete but: 20 megabit/mm^2 for SRAM...a 1u rack is 600mm X 914mm = 548,400mm^2. Multiply that by 20 megabits and that is about 70 Gigabytes. Does that mean in theory we could build a rackmount server with an external L1 cache of 70 Gigabytes? The cost would be horrendous but I'm sure there is a scenario where it could make sense.

This would require an impractical amount of wires. For an 8 core, 64 bit cpu with differential signaling would need something like 8(64+64)2 = 2048 wires, and the length of the wires would mean the latency would be much worse then an on-die cache.

No, I believe SRAM actually has the edge in that department, and by a fair margin too. The d in DRAM is for dynamic as contrasted with S for static. DRAM need to be constantly read and refreshed (the timings) while SRAM doesn't and that comes with a pretty big hit to energy consumption. As soon as you add DRAM to an embedded project, the thermal/power envelop increases.

Of course that all depends on the generation of tech and only applies in an apples to apples scenario.

This is why even though memory was becoming dirt cheap, prices started to spike in the last few years. There are also less manufactures now as a result of buyouts and mergers.

Add the European Union, Infineon.

[0] https://www.finanznachrichten.de/nachrichten-2010-05/1694023...

I'm not an expert, but my understanding of it is that DRAM is mostly just banks of capacitors with some transistors to control when they get recharged or drained. Making capacitors smaller is problematic, because it would decrease the capacitance and require more frequent updates to prevent data loss. So, DRAM doesn't really benefit as much from process shrinks as CPUs do. DRAM does get better and cheaper over time, but the physical constraints on the problem are different.

A DRAM cell is a single capacitor and single transistor. DRAM benefits from process shrinking because smaller transistors require less charge to switch. The way DRAM is read is by dumping the charge into a transistor’s gate capacitor. Yes, you also have to deal with leakage current and stray capacitance and the like.

You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. Existing parts can address 4TB. x86-64 has an ultimate system limit of 256TB, due to its 48-bit virtual address space.

Also worth considering that 32TB of DRAM would draw over 12kW, just sitting there.

Ice Lake added 5-level paging and 56-bit pointers, so they can address 128 PB: https://en.wikipedia.org/wiki/Intel_5-level_paging

Hey, I'm a bit late, but it's 57-bits. On 64-bit systems page tables have 512 = 2^9 entries and page frames are 4096 = 2^12 bytes long.

So the number of bits in an n-level system is 9n + 12.

9x5 + 12 = 57

On 32-bit systems the number of PT entries was 2^10, which is why they could have a 2-level system. 10x2 + 12 = 32.

> You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. Existing parts can address 1TB (Intel) or 4TB (AMD). x86-64 has an ultimate system limit of 256TB, due to its 48-bit virtual address space.

That's the virtual address space. A page table entry has enough bits to have a 64-bit physical address space, it just wouldn't be able to have it all mapped at once in the same virtual address space. Although CPUs don't have 64 physical address lines yet, there's nothing fundamental in the x86 architecture preventing them from doing so.

Intel have already implemented 5-level paging, which would give a 2^57 bit virtual address.

Also, this wouldn't be the first time that the x86 has supported more physical memory than virtual. PAE allowed for 64GB of RAM on a system with a 32-bit virtual address space.

It costs them money to add physical address pins, though.

While we're here I hope anyone can explain to me why Cooper Lake Xeon parts are listed as supporting 4.5TB of DDR4.

Answering self: the 4.5TB support is mostly Optane memory, not DDR4.

The Google Cloud machine type m2-ultramem-4164 has 11,776 GB of memory. How does it accomplish that then?

4 sockets?

https://yourdatafitsinram.net/

IBM Power System E980, max ram: 64TB.

Sure but it only supports that with 16 CPU sockets.

You need a CPU that doesn't exist yet just because it's DDR5!

Expanding from 48 toward 64 bits isn't difficult.

Whoa! 12kW?? Just from RAM? I didn't realize RAM drew so much power.

Crucial says approximately 3W per 8GB: https://www.crucial.com/support/articles-faq-memory/how-much...

  $ echo '(32 * 2^40) / (8 * 2^30) * 3' | bc
  12288

For what its worth, a significant fraction of that is in the communication interfaces, and not from the ram itself, and there have already been significant process improvements to reduce ram power consumption. A modern 256GB RDIMM draws a heck of a lot less then 50w--I have never measured but based on the thermal solution I would say closer to 5w

I don't see how that could be true. On my server right here with a Xeon Silver 4114 it is measuring the power consumption of the memory at ~75W for 256GB.

Like I said, ram power consumption does not scale linearly with capacity due to the significant overhead from the IO. A single 128GB stick will draw much less than 16x16gb sticks (not sure why you are using 256GB on a 4114, it has 6 memory channels so surely you have 288GB?)

Here is the datasheet for a 128GB dimm from 2017 [1], which shows 3.4A IDD0 (normal operation) on the 1.2V rail at the highest speed of DDR4-2666, and 0.2A on the 2.5V precharge rail for a total of just over 6W. Also worth noting is that is a LRDIMM, which draws more power from the DC rails due to the additional buffering. A normal RDIMM draws a bit less static power.

Compare to a manual for a similar vintage 32GB stick [2], which consumes 2A on the 1.2V rail and 0.1A on the precharge rail for a total of a bit under 3w. One quarter the capacity, but still half of the power draw.

[1] https://www.samsung.com/semiconductor/global.semi/file/resou... [2]https://static6.arrow.com/aropdfconversion/d3b3ce1d78b0ad7d3...

If I could send you back in time to stop this machine's designer from deploying it at scale with some of the channels depopulated, I would! It's an HPE DL360 g10, if you go look at their catalog you'll see that all of the off-the-shelf and BTO memory configs are nonsense.

Thanks for doing the math on the power story. I didn't realize about the scaling.

> You would need a new CPU that doesn't yet exist to address 32TB of memory per socket.

I thought RISC-V could with the RV128 ISA.

> How is Intel Optane going to compete?

You should see how fast Optane will copy a project folder full of node_modules, bin, etc.. Blows the doors off an Evo Plus.

That said, probably not worth the premium for the vast majority of uses.

I assume you're talking about Optane SSDs, not Optane PMem, given the Evo Plus comparison.

You can avoid that problem by not using node and npm. I have used create-react-app once for a project and that was bad enough for me to stay away from the entire ecosystem.

> You can avoid that problem by not using node and npm.

Not realistically; that would just introduce more problems than what I would avoid.

I have better things to do with my time than try to avoid npm.

JS projects are only a small fraction of my work anyway.

Is anyone here in the industry? Is there any new tech or process on the horizon that could bring down the price of DRAM?

Tech or process hasn't been the main reason for high DRAM progress for a long time.

You can do static RAM this way eg cache but off chip is dynamic ram which is a slightly different process as it uses very small capacitance to store the charge, I also think mixing the two processes on one die is the issue cache is so small relative to dram

you can already have 128gb per DIMM with DDR4. not sure if its up to spec, but it works usually.

128gb dimms are more than double the price of 64gb dimms, so it not always economically viable unless you need max memory density.

> Intel Optane going to compete

Price ;=)

Also Optan drives (over PCI) are probably still interesting for some applications due to low latencies as far as I remember.

Part of it is that the demand isn’t there for extreme densities. Producers are focussing on speed.

Look at base ram in MacBooks, the growth over time is pretty slow.

To the best of my knowledge: with (non-volatile) NAND; 1ms response times vs (volatile) DRAM; 1ns response times... For context; there's 1 million ns in 1 ms. They're in completely different leagues in terms of speed and they're used for very different applications.

While both benefit greatly from economy of scale; the manufacturing tolerances, equipment, etc etc used influences pricing; but I'm not an engineer... so maybe someone here can chime in on that :)

Those numbers seem awfully pessimistic for NAND and too optimistic for DRAM. 1ms is what you get from a fast HDD. SATA SSDs usually manage 100µs and NVMe usually has 10µs or less latency (as low as 2.8µs). DDR4-3600 usually has around 10ns CAS latency which only measures the time needed to read data from a DRAM cell but doesn't consider how much time it takes for that data to travel to a CPU. Before a CPU looks up data it has to first check the L1, L2, L3 caches which can take 30ns and then another 20ns for the memory controller to process the DRAM request. So it's 60ns (10+30+20) for a main memory access.

The difference between 10µs and 60ns is merely a factor of 167 not 1 million.

The only take away I have from your comment is that you somehow confused DRAM with the L1 cache and SSDs with HDDs. That's the only way one could possibly arrive at your numbers.

I have used rounded numbers for illustrative purposes. They might be off by 30% or more but they are within the right order of magnitude.

Reply on news.ycombinator.com