Nvidia Criticizes AI PCs

While they are more performant still, nvidia tries to lock customers into artificially expensive products.

RTX consumer cards suck at AI because they simply lack the memory. That is where even an Macbook Air with passive cooling and max memory config is nearly as capable as an RTX4090, the larger Macs plainly beat everything nvidia puts into consumer tech because they can load larger models.

But sure, smaller models run faster on nvidia cards and the difference is still very noticeable. But complex models fail because nvidia doesn't have enough memory on their consumer cards. If they don't change here, other hardware will prevail. If there are similar optimizations that currently exist for cuda, they might already be on par.