Hacker News Reader

For general purpose use this is interesting, but if I'm just using an LLM for coding, does this matter at all? I would hope something like a new java version after a model's publish date can be handled and understood by the model through tool calls and context even if it's not explicitly in the training data, the same way the LLM doesn't have my existing code or the plan to change it baked in from training.

View on HN

The inference time results are quite different. These models haven’t just been trained with tons of docs, blog entries and videos about Java 8. They’ve also been trained on tons of code examples from simple to complex real world ones.

You’re not going to get all that just by pointing the LLM to the recently released Java 27 documentation. That information is also potentially adding tons of content to your context, which is already filled with tons of other data (your code, other recently released libraries it has to get documentation for, etc).

Gotcha, that makes a lot of sense. Thank you

My experience with Claude is that it doesn’t handle its own training cutoff properly. It responds as though its cutoff is today. Gemini is even worse about this.

I’m going to add this site to my agents files so that they’re explicitly aware of their own limitations.

Reply on news.ycombinator.com