I want to run Stable Diffusion at stupid resolutions, and I want it cheap.
There would certainly be an awkward silence in response from Nvidia.
Given that Intel needs some angle to get ahead in the GPU wars I would have thought adding RAM would be an easy way to do that - especially given Nvidia and AMD seem to dish out RAM like its the most precious substance in the universe.
Well, it's only really nVidia. And the reason it's only nVidia doing this is because they don't want to self-cannibalize; if the 4080 had 24GB literally no prosumer or small organization wanting to play with the tech would even be considering the 4090 (and to a limited extent, the same is true of the 3080 vs. the 3090, but everyone was too busy mining Ethereum to notice that).
As far as the other manufacturers are concerned, there's no objective need for anything over 16GB on midrange cards (2070/3060/4060 equivalents) because that's the kind of GPU in the consoles and nobody releases exclusively for PC any more; you'll have to upgrade your card to have parity when the next generation comes out anyway so why provide more/drive costs up? It's not like AMD or Intel GPUs are in any way able to use that RAM for anything else, and why offer it now and kneecap themselves should they somehow get into a position where they actually can offer a good AI suite and want to sell 24GB cards for 2000+ dollars too?
And nVidia has taken full advantage of that fact, and that's why the 4080 only has 16GB even though the 7900XTX is performance-competitive with it and has the expected memory compliment. The 4080, including the Super, is a sub-par product because it kind of has to be.
Great quote, and apt description of this mildly infuriating trend.
I'd also imagine that the hardware on the smaller/cheaper dies doesn't have an address bus wide enough for 128GB.
Not exactly tiled mind you, just "clearly repeating the same thing, with some variation".
Asking because when I was playing with it about some months ago, that's what it did. People explained (at the time) that it was because stable diffusion was trained on um... (from rough memory) 512x512 images.