Hacker News Reader

Comment by aenis | Hacker News Reader

Good news for Intel and AMD.

Rught now on large scale codebases the bottleneck is both claude/codex inference, as well as time it takes to run tens of thousands of tests. We put those workloads on dedicated epyc 9005 build machines - but it still takes minutes per run. Those who can afford the fast tokens will be in the market for faster CPU that money can buy today.

View on HN

why intel and amd ? these are cerebras wafers?

i know people are joking about the sol ultrafast prices (its unlikely to be accessible for average joes) but this shows scaling wafer cores works for inference boost

which makes me very excited, sol ultrafast will be as slow as it will get if that makes sense. at these token speeds , we will see a much deeper economic impact.

Right now the bottle neck is not the CPU, so people aren‘t spending big $ on them. But with this ultra fast mode, CPU becomes a bigger part of the bottleneck and thus Intel and AMD can charge more $$$.

I noticed it myself. It's not just that the models are faster, but that by becoming more capable, they can take on larger tasks, which require larger compute.

I am settling on a way of working where during the day I spent the time iterating requirements, designs and low level implementation plans for, say, a dozen features, and then I park them for an unattended build during the night. It works well.

On large codebases, literally anything I throw at Fable or Opus takes at least 20 minutes, 10 of those minutes is spent on running test suites. With this way of working, I put way more emphasis on test coverage, integration tests, behavioral tests, in-browser tests -- and I run them on every change, before any commit. Faster CPU solves this, while faster inference solves the other 10 minutes. I can't imagine how cool it would be to work with this ultrafast mode, without those mental-context-purging pauses.

gotcha, we got faster inferences but now CPU has to deal with the tasks.

Reply on news.ycombinator.com