Hacker News Reader

Comment by CharlieDigital | Hacker News Reader

No one mentioning Decisions API?

Feels huge. Sounds like Luna on their Ultrafast infra.

I did some internal benchmarking and found that GPT 6 Luna in batch mode at 20 records per batch was about 1.6x faster at 1.2x the cost of Jev with no tradeoff in accuracy. Tuning it up/down would make it faster per-record while also reducing the cost (assuming accuracy holds). Though this was only useful for offline processing.

View on HN

It's another Jev copy, like we've seen so many over the last few weeks. But with no benchmarks or price comparison, which likely means it doesn't compare that well.

    > But with no benchmarks or price comparison, which likely means it doesn't compare that well.

Doubt that's the case; this was only the announcement and the API isn't even released yet. As I noted, my own testing using a batching strategy (nothing more than "read 20 lines from this file instead of reading 1 at a time") yielded ~the same accuracy on an 800 row dataset we have with ~1.6x the throughput (at only batch_size=20) and 1.2x the cost. Didn't tune to find accuracy dropoff by batch size, but it's easy to see that the decision capability was there (understood that it's not the same without the probabilities, but even without it, the classifications were nearly identical).

Reply on news.ycombinator.com