Comment by ly3xqhl8g9 | Hacker News Reader

Comment by ly3xqhl8g9 | Hacker News Reader

Parent Full thread

Until now there wasn't much incentive for a consumer to have 1TB+ of RAM, for any CPU or GPU task: what were you going to do with that much RAM? render 1,000 more Chrome tabs?

Beyond the supply/demand economics and also corporate greed [1], there are fabrication issues, just look at the current mess [2], but as new US-based fabs come online from 2025 and beyond [3], they will be solved to some extent.

Theoretically, the maximum amount of RAM a 64-bit machine can address is around 16.8 million terabytes [4]: as always with computing, the journey hasn't even begun.

[1] Easier and more money in manufacturing RAM for smartphones, https://fossbytes.com/flagship-phones-ram-vs-pc-ram

[2] "EU tells Taiwan to forget about a bilateral investment pact even as bloc seeks more chips", https://www.scmp.com/news/china/article/3213163/eu-tells-tai...

[3] "U.S. Semiconductor Renaissance: All the Upcoming Fabs" (2022), https://www.tomshardware.com/news/new-us-fabs-everything-we-...

[4] https://superuser.com/questions/65191/what-is-the-maximum-am...

There isn't much incentive for a consumer to have 1TB of RAM in 2033 either.

AI models that require a huge amount of memory will be done in the cloud.

Will memory demand increase by 2033? Of course. Will demand reach 1TB of RAM for a consumer level computer/mobile device? No.

Yeah, but I think it might legitimately 5 or 10x, if running models on-device is as helpful as it sounds.

> Will demand reach 1TB of RAM for a consumer level computer/mobile device? No.

Speculative Sampling allows to run a smaller model locally and only check up with a larger model in the cloud from time to time, once every few tokens.

There is incentive to use RAM as planned obselence though. As can be seen when updating old devices, forcing apps to swap frequently in and out of disk

I think nowadays Intel CPUs are more planned obselence that any other hardware, so behind.

Today the upper range of RAM in devices is in the hundred-GB. 10 years ago it was about 16GB or so, and making a (probably bad) interpolation I think it wouldn't be crazy to have the upper range in the TB mark in 10 years time. We could get there faster too, for specific use cases (compiling, rendering, etc).

On the other hand, yes, everyday use (web, mail, video/media consumption) don't require today much more than 8 or 16GB of RAM. If we go a bit creative, a PC with Linux can run very smoothly on 4GB alone, and surely someone here can point to their one anecdote of a machine with 2 GB or even 1 GB sporting a nice desktop environment, or 128mb CLI-only machine.

Edit: also memory has to improve its bandwidth and data transfer rates to keep up with faster processors, so it could also improve over 10 years time without much focus on storage capacity. Or maybe they focus on latency instead, or a mix of all three. Point is that it's not a single metric to improve.

Reply on news.ycombinator.com