Hacker News Reader

Comment by Der_Einzige | Hacker News Reader

To this day, Intel is still deeply ahead in software support for AI.

MKL, OpenAPI, Intel Optimization for sklearn, for pytorch, etc (there are many more). No equivilants exist from AMD but NVIDIA has an equivalent for all of these (i.e. CuPy).

Using AMD for AI stuff only makes sense if you never believe that you may want to do any kind of ML-like workload on or with the CPUs assistance. This is true for many folks though, but for those it's not true for, they will buy blue until Intel no longer exists.

View on HN

MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's still due to shared memory with GPU).

I'm doing fast AVX512 embeddings on Ryzen, and fast ONNX AVX512 reranking on Ryzen. Though I do the actual heavy lifting on GPU, doing all the RAG stuff in CPU is helpful. AI on CPU is still mostly a gimmick, but as models get smaller and more capable it's becoming less of a gimmick.

Yeah, but the "AI" on CPU means some basic NPU doing (sparse) matrix multiplication, not AVX512.

I'm handling dense, sparse, and colbert vectors generated by BGE-M3 model on CPU, pretty sure that counts as "AI"

Certain workloads aren't total "gimmicks" (i.e. small embeddings models), and often those CPU optimizations are on things that absolutely do take advantage of the CPU well (i.e. graph analysis) or on traditional ML algorithms like random forests (intel optimizations for sklearn) which are still important and do still run well on CPUs.

Not to mention that most people have more RAM than they do VRAM.

Running MKL on your datacenter grade AMD processor technically violates some license and it relies tricks like this - https://danieldk.eu/Posts/2020-08-31-MKL-Zen

Technically, if your found to be allowing this as a cloud provider by Intel, they have grounds to sue you, so yes, you can use MKL on a Ryzen at home, but you put yourself at risk for lawsuits by doing this in a data center at scale

Reply on news.ycombinator.com