consumer RAM is not what's creating shortage. Data centers doesn't run electron to train the model or for inference
A human doesn’t need 100TB of books to learn the alphabet.
A human does need 16ish hours per day of audio/video content for several years to learn the alphabet.
Living inside a normal home with my parents was enough for the audio part.
It was not meant as literally sitting at a screen with audio/video for 16 hours a day.
IOW, humans still learn more effectively with less information, because there are innate mechanisms which process this data continuously and extract new meanings from the same data. This is part of both intelligence and consciousness.
LLMs lack both.
> because there are innate mechanisms which process this data continuously and extract new meanings from the same data
To me, these statements strongly contradict each other, but I also really do not care enough to debate it.
Have a nice day.
Is that really the case? How much data is it for 4k video, high bitrate auditory, spacial mapping, internal and external nervous system, emotions, and a dataset to correlate all of these in time?
So, getting systems with higher RAM capacity is getting harder (from laptops to smartphones). So, for a couple of years, we need to stop using Electron so much and use what we have efficiently.
Data centers, esp. AI hyperscalers do not care about efficiency for now, because they can suffocate consumer-grade part of the hardware marketplace and get anything and everything they want. When their bubble pops, or the whole capacity ends, they need to learn to be efficient, too.
For reference, a well-optimized cluster runs at ~90% efficiency even though they have thousands of users. AI hyperscalers are not there. Maybe 60% efficient, at most. They waste a lot of resources to keep their momentum.
https://v-color.net/products/ddr5-ecc-oc-u-dimm-server-memor...
But they no longer have 6000mhz stuff in stock (which is ideal for Ryzen due to the 1 to 1 speed match to the memory controller).
It's frustrating :(