I have a dim sense of the hardware industry, but I know that disk storage has gotten 10-100x cheaper, CPU/GPU power increases quite steadily, but RAM is still quite expensive.
Is it even possible for RAM to get much, much better than it is currently? Are there physical limitations, or is there just not the same amount of market interest in more RAM?
Beyond the supply/demand economics and also corporate greed [1], there are fabrication issues, just look at the current mess [2], but as new US-based fabs come online from 2025 and beyond [3], they will be solved to some extent.
Theoretically, the maximum amount of RAM a 64-bit machine can address is around 16.8 million terabytes [4]: as always with computing, the journey hasn't even begun.
[1] Easier and more money in manufacturing RAM for smartphones, https://fossbytes.com/flagship-phones-ram-vs-pc-ram
[2] "EU tells Taiwan to forget about a bilateral investment pact even as bloc seeks more chips", https://www.scmp.com/news/china/article/3213163/eu-tells-tai...
[3] "U.S. Semiconductor Renaissance: All the Upcoming Fabs" (2022), https://www.tomshardware.com/news/new-us-fabs-everything-we-...
[4] https://superuser.com/questions/65191/what-is-the-maximum-am...
AI models that require a huge amount of memory will be done in the cloud.
Will memory demand increase by 2033? Of course. Will demand reach 1TB of RAM for a consumer level computer/mobile device? No.
Speculative Sampling allows to run a smaller model locally and only check up with a larger model in the cloud from time to time, once every few tokens.
On the other hand, yes, everyday use (web, mail, video/media consumption) don't require today much more than 8 or 16GB of RAM. If we go a bit creative, a PC with Linux can run very smoothly on 4GB alone, and surely someone here can point to their one anecdote of a machine with 2 GB or even 1 GB sporting a nice desktop environment, or 128mb CLI-only machine.
Edit: also memory has to improve its bandwidth and data transfer rates to keep up with faster processors, so it could also improve over 10 years time without much focus on storage capacity. Or maybe they focus on latency instead, or a mix of all three. Point is that it's not a single metric to improve.
DRAM currently consists of a small capacitor per bit (cell), which can hold a small amount of charge (1) or not (0). The amount of charge it can hold, its capacitance, depends on the area and the distance between the conductors[1].
So, as you try to cram more cells into the same area, the amount of capacitance goes down. This is problematic because of leakage[2], causing the charge to leave the capacitor. This is why DRAM cells are refreshed periodically, hence the "dynamic" in DRAM. Too much leakage, and you spend all the time refreshing rather than accessing the memory.
To raise the capacitance back up, you can move the conductors closer. However, if the insulating material stays the same, this will lead to increased leakage. Thus a better insulating material is needed.
While the current DRAM cells are quite impressive, it seems they're starting to hit fundamental road blocks with the current technology[3]. There's a limit to just how little amount of charge a cell can hold and still be reliably read, and finding better insulators is getting harder and harder.
It may well be a new approach is the solution to continued scaling. Moving away from individual capacitors as the basis for a memory cell might be a way forward[4].
[1]: https://en.wikipedia.org/wiki/Capacitance#Capacitors
[2]: https://en.wikipedia.org/wiki/Leakage_(electronics)#In_capac...
[3]: https://www.techinsights.com/blog/dram-scaling-trend-and-bey...
[4]: https://spectrum.ieee.org/new-type-of-dram-could-accelerate-...