Hacker News Reader

Thought they were gonna go CPU based inference with llama.cpp/ggml, this project should definitely be made into a programming book

I think they quickly reached the limits of CPU inference and that's why they decided to scale up with GPU's.

Because of how they operate, they can't just release in a single DC like other providers. It's a partial "all or nothing" scenario.