very interesting. Do you know of anything that would take advantage of the 128 cores of my Ryzen Threadripper even though I have a 2080 and a 3080 as well? (Or all three... lol)
Even if you could use all 128 cores I would still get the 2080 would be faster
https://www.pcworld.com/article/402592/nvidia-turing-gpu-gef...
IMHO these chips designed for specialized workloads are looking more and more ridiculous. I expect GPU manufacturers to keep dragging them out for another decade or so, as well as Apple as it explores offshoots of M1. It all makes me feel very tired (the ultimate code smell).
A better design would be something like a 256+ core RISC-V with local memories in/by each core for data-locality and a content-addressable caching scheme for deduplication. Copy-on-write languages like Clojure and orchestrating processes under something like Docker would make it a breeze to program, although it would still support manually managed mutability like with Rust for innermost loops in games or whatever. It's fairly obvious how it would all work, but IP law and gatekeeping ensure that it will not happen anytime soon.
Then stuff like CUDA would be just another framework run on a symmetric multiprocessor and we could get back to exploring alternatives like genetic algorithms like we did in the 1990s. Thankfully nobody cares what I think, heck even I'm sick of reading my own complaints, so it's easy enough to just unsee this.
I mean, “many” is not usually the case; an RTX 3090 has 10,496 CUDA cores, 328 Tensor cores, 82 RT (raytracing) cores, and 96 render output pipelines. ML apps will uses the first and, depending on software, second set. The vast majority of the cores being CUDA cores is the norm.
* The largest contributor is the sheer number of cores.
* The latency between cores and memory.
* FP16 instructions.
128 cores isn't an insignificant fraction of the number of cores on a 1050 (about 600), but CPU cores are individually more powerful. That advantage is potentially difficult to call. The top-end of Genoa has 96/192 cores, and you can slot many of them onto a single board.
AMD is throwing more and more memory into the CPU cache. That's very different to a direct path to GBs of HBM, but at some point the difference in performance might not matter to a novice/dabbler.
I would say give it a go, see what happens.
ML is really something which should be left to a GPU.