Hacker News Reader

I’m confused as to why you think a GPU is necessary? It’s just linear algebra.

most likely he was referring the fact that you need plenty of GPU-fast memory to keep the model, and GPU cards have it.

There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.