Hacker News Reader

What do you run it on? And even then, I'm guessing your tokens per second are not great?

I get about 35-40tok/sec on a 3090.

It's actually about the same speed when accounting for how much more responsive my system is to Anthropic's saas infrastructure

I keep forgetting have a 3080 laying around... Gotta figure that out.