Hacker News Reader

This comment makes so much sense relative to what I've seen with Claude's 1M context window. It reliably fails to succeed a task with a prompt where I just stuff in a big blob of data in the middle as context. But when I use emebddings to only select a small relevant subset of that data, it always passes the task.

View on HN

Yes, Claude 1M is using all sorts of approximation tricks to get that 1M context window. IMO this is actually quite deceptive marketing.

Yes, that's the point now for competing AI research-for-profit companies, whatever metric is technical and sounds important, is going to be used in marketing and valuation determinations. It will be explored for research I'm sure, and then determine its product viability. It's nice competition, but agree, that it can be deceptive.

Claude's context is 100K not 1M [1]. If you're somehow shoving in a million tokens that could explain the issue you're having!

[1] https://www.anthropic.com/index/100k-context-windows

Misremembered, the main thrust of the comment still stands, the 100K context window isn't "real", it would be absurdly expensive to do it for real. They are using a lot of approximation tricks to get there.

Yep, mistype on my end as well. Claude just fails to process the request if you get above 100k tokens (I've done that, heh).

Reply on news.ycombinator.com