Hacker News Reader

> We first struggled with token limits [solved]

How has this been solved in your opinion? Do you mean with recent versions with much bigger limits but also heaps more expensive?

View on HN

The limits still exist but for certain use cases larger limits have helped

Most of the very large token limits are just fake marketing bullshit. If you really try them out, you will immediately realise that the model is not at all able to keep all 100k tokens in its memory. The results tend to be pure luck, so in the end you end up just using 16k tokens anyways, which is already much much better than the initial 4k, but still quite limiting.

Reply on news.ycombinator.com