Hacker News Reader

What’s the expected additional latency due to running this re-ranker?

It actually runs pretty fast, our benchmarks show ~149ms for 12665 bytes. It's faster than many other models

I would prominently display your benchmarks (against your competitors, of course). That's your selling point, right?

Yes! We did this here: https://www.zeroentropy.dev/blog/announcing-zeroentropys-fir... We wanted to share the approach with the community in this post. It does do better than competitors though!