Hacker News Reader

14 / 20 points. 19 canary tokens reported. 2 /4 qualitative points.

On pi-coding-agent, with my pi-web-browser extension and glm-5.

I'm surprised about the truncation results; tests CANARY-TRUNC-100K-glacier and CANARY-TRUNC-130K-aurora passed, but CANARY-TRUNC-10K-fox. CANARY-TRUNC-40K-river and CANARY-TRUNC-75K-summit failed.

View on HN

I suspect this is a result of relevance-based retrieval. In my colleague's testing, they found that sometimes the content comes back out of order or not at all, depending on the implementation's interpretation of which chunks of content were "relevant" to the query that accompanied the fetch result. I was surprised to find out some agents do this - when I started down this rabbit hole, I assumed they either returned some number of characters in order or did some sort of summary.

So many different implementations out there!

Reply on news.ycombinator.com