Hacker News Reader

> with no noticeable differences in quality or velocity.

Have you done any A/B tests on this with evidence? (That's one thing I'd be very interested to see for claims like this - I'm not necessarily doubting you, it just seems like it could be useful to understand claims of quality/efficiency)

View on HN

Great question! Our main product quantifies engineering productivity & quality so I think we're uniquely qualified to answer this - our velocity has only gone up and our quality (bugs introduced, code turnover) has not budged per our own analysis.

> our velocity has only gone up

That is super curious - using more low quality cheaper models increased your velocity? My prior would have been slightly reduced velocity but massive reduction in token costs made it worthwhile.

Is that due to the faster inference time?

Reply on news.ycombinator.com