Hacker News Reader

Comment by zuzululu | Hacker News Reader

why intel and amd ? these are cerebras wafers?

i know people are joking about the sol ultrafast prices (its unlikely to be accessible for average joes) but this shows scaling wafer cores works for inference boost

which makes me very excited, sol ultrafast will be as slow as it will get if that makes sense. at these token speeds , we will see a much deeper economic impact.

View on HN

On large codebases, literally anything I throw at Fable or Opus takes at least 20 minutes, 10 of those minutes is spent on running test suites. With this way of working, I put way more emphasis on test coverage, integration tests, behavioral tests, in-browser tests -- and I run them on every change, before any commit. Faster CPU solves this, while faster inference solves the other 10 minutes. I can't imagine how cool it would be to work with this ultrafast mode, without those mental-context-purging pauses.

gotcha, we got faster inferences but now CPU has to deal with the tasks.

Right now the bottle neck is not the CPU, so people aren‘t spending big $ on them. But with this ultra fast mode, CPU becomes a bigger part of the bottleneck and thus Intel and AMD can charge more $$$.

I noticed it myself. It's not just that the models are faster, but that by becoming more capable, they can take on larger tasks, which require larger compute.

I am settling on a way of working where during the day I spent the time iterating requirements, designs and low level implementation plans for, say, a dozen features, and then I park them for an unattended build during the night. It works well.

Reply on news.ycombinator.com