Hacker News Reader

Are you developing software? Is most of it used on a coding agent? (Like Claude Code or ChatGPT Codex?) If so, what coding agent do you use? If you're not developing software what do you use it for (roughly)?

Our use cases are data analysis, software development, general AI chat with RAG so far.

> We're running Kimi 2.8 on a $107k server

Equipped with what? Is it CPU based inference, a mix...?

Not GP, but my educated guess is that they are running a system with between 4 and 6 MI325 or MI355x or similar AMD GPUs. With the 50k tps as the total figure for all parallel requests. Those cards have a lot of memory for their price, allowing you to push to really high batch sizes while still having a large context size for each request

4x MI300A in a Gigabyte server

What is your config?

how many requests per second can the server take?

That's about it across the whole server. Kimi 2.8 is about as big as we can go. Kimi 3 is massive.

on 1 paralel request maybe

Correct but even with 30 simultaneous users, we're not seeing any noticable lag compared to AIaaS even with agentic workloads.