Apple M series are competitive in inference at least. I wish Apple would just aim their chip people at NVIDIA in everything else. They are probably the only ones that have the talent, resources, and capital to do that.
Even for home inference, it's hard to recommend a dedicated Mac over a cheap Nvidia server box.
> They are probably the only ones that have the talent, resources, and capital to do that.
Apple invented OpenCL. The problem was their reluctance to work with the rest of the industry, and once CUDA took over it was too late for them to even try.
NVIDIA hampers their GPUs with un-unified graphics memory, while the M series can use everything the computer has (well, you need to save 4GB or so). It also works on airplanes and in hotel rooms, a cheap NVIDIA server box with 64GB of RAM (what my M3 Max laptop has)....how cheap is that?