Isn't that exactly what's happening? From OP:
> The GeForce RTX 2080 Ti 22GB has found its way outside China. eBay seller customgpu_official, an upgrade and repair store in Palo Alto, California, sells a similar MSI GeForce RTX 2080 Ti Aero with 22GB for $499. The store is touting the graphics card as a budget alternative for students and startups that want to get their feet wet in AI workloads. The graphics card is allegedly stable in Stable diffusion, large language models (LLMs), and Llama 2. According to the merchant's website, it has sold over 500 units of the GeForce RTX 2080 Ti 22GB.
The China stuff is just background story I think.
> Nvidia's continuing refusal to put higher amounts of VRAM into its "affordable" cards [...] leaves average consumers who want to run bigger models locally with very few options.
That's market segmentation and product differentiation for you. You want that much VRAM you're doing ML; you want to do ML you can pay us ML prices.
What 'average consumer' wants to 'run bigger models locally' anyway!
If they made the 'affordable' cards with VRAM ranging from say 8GB to 80GB, it would be only gamers buying at the bottom end and only MLers at the top. And they can charge the latter a lot more - so even if they did what you want, you'd end up with Applesque pricing bumps for beefier/more GDDR chips.
Boards from different OEMs were different in meaningful ways and OEMs had the chance to differentiate themselves from the competition with some actual engineering. More RAM, multiple GPUs per board, AGP to PCIe chips, you name it.
Nowadays Nvidia restricts is partners from all that fun and undercuts them with their own models. No wonder EVGA quit this game.
Usually by the time you could buy a high-end CPU replacement, it was a "better deal" to just replace the entire motherboard/CPU combo and get a whole new computer. But a second video card, some more RAM, additional SSDs, those made sense.
Games are just going to use temporal accumulation now. There is too much signal to be gained from re-using past samples. It makes things way cheaper and opens the door to extrapolation and non-uniform sampling etc. It's the least bad of all the options, and DLSS actually is quite good at weighting samples pretty reasonably. DLSS 2.5, 3.0, 3.5 and upwards are actually significantly better and that can be injected back into (non-anticheat) games and the bar will likely continue to be raised. It is a signal processing technique that recovers a lot of signal with very low "noise factor".
A few generations ago (1080/2080) there was no mainstream ML. Products change, product lines change.
Care to elaborate?
The RTX 4090 is so huge and power-hungry you can barely fit two into a case, let alone any more than that.
And even if you replace your case, and your power supply (and maybe your motherboard and CPU too, gotta have enough PCIe lanes) you'll still only have 48GB of vram.
OK, so we can't use an RTX GPU. Good luck even understanding the rest of nvidia's product range. The A40? "The World’s Most Powerful Data Center GPU for Visual Computing". V100? "The most advanced data center GPU ever built". H100? "Unprecedented performance, scalability, and security for every data center". A100? "Unprecedented acceleration at every scale to power the world’s highest-performing elastic data centers". H200? "The world’s most powerful GPU" L4? "The breakthrough universal accelerator".
Good luck figuring out who sells them in your country, or if they're in stock.
Oh, and remember to read the entire spec in excruciating detail. That $2700 L4 24GB is slower than an RTX 4090.
Think you'll launch something in the cloud? With Google Cloud half the cards are only available in some regions - and even if you're in a region that offers a given GPU, maybe there's a shortage and support tells you to just keep requesting GPUs repeatedly until you get lucky.
I had to upgrade to a 1000W PSU for the 3090 because the 3090s will occasionally voltage spike and trigger OCP on power supplies. TDP is supposed to be like 350W but for a slip second it might pull around 600-700W. Plenty of complaints about it on Reddit and elsewhere. It’s a beast but it cooks my home office when I use it. Cest la vie.
* Is the product actually available for purchase today, and at what price?
* In what form factors - Full height? Half height? Two slots? Three? SXM? Does it have a fan built in?
* What is the relative performance? Half the time the specs all quote different numbers. Why does one product quote 'single precision performance' and 'rt core performance' and 'tensor performance' while another quotes the 'tensor cores' and 'shader cores' and another quotes the 'FP64', 'FP32', 'FP16', 'BFLOAT16', 'TF32' and 'INT8'?
And don't imagine you're going to get away with ignoring those specs. A $2700 L4 24GB is slower than an RTX 4090, for example, because it's for power-efficient servers or something.
Like I said Nvidia themselves indicate whether it’s suitable.
And if your still confused or doubt the accuracy, or don’t want to spend any time reading spec sheets, I imagine the sales channel folks will be able to guarantee it in writing for a fee.
You'd think so, wouldn't you?
But nvidia is not that smart. They have decided that the GPUs should be split over at least three different pages, and those pages should be camouflaged.
I can only assume the marketing team are judged based on time spent on site or number of pages viewed, rather than on sales made.
When you go to the home page, point to products and click on "NVIDIA RTX / Quadro" you might expect to find the RTX 4090 and some Quadro products. You will in fact find neither - the RTX 4090 is under 'GeForce' and the 'Quadro' brand is no longer used.
Maybe on the home page you choose "NVIDIA RTX-Powered AI Workstations" - that'll give you a list of their workstation-suitable cards for ML, right? No, that page contains no products at all. The only option is to Find A Partner which links you to a bunch of partners - several of whom do not in fact sell AI Workstations.
Other partners sell AI workstations... without GPUs. As far as HP is concerned, $4000 only gets you the base model workstation, with 32GB of RAM and a 1TB hard disk. If you want GPUs with that, you'll need to call the sales team, who might perhaps deign to sell you one.
Or perhaps you're at nvidia's home page, and you're looking for data centre grade GPUs? For that product page, simply choose whether you mean the DGX, EGX, IGX, HGX, MGX or OVX platform?
But they make it very easy to find the keynote speech by His Excellency Omar Sultan Al Olama on the latest breakthroughs in AI. He does, in fairness, have a very appropriate surname.
What, exactly, is your expectation here? If you want the simple Apple model where you don't need to look at the spec, do the Apple think and pay up for the pricy GPU.