Hacker News Reader

Comment by jonathan-adly | Hacker News Reader

I didn't really trust Google's 1m+ context. But I trust this paper and 1m+ context would be available everywhere.

I know people complain about hardware and compute resources. But, this is like complaining about Python resource usage in the early 90's. The development complexity & resources is far more expensive than chips on the long run. I am personally re-organizing my AI organization to move away from complex RAG setups and get comfortable with long-context workflows.

Just to be clear - I also think that inference-optimized chips are the next frontier - Nvidia GPUs were designed & built in a different age than what’s going on now.

View on HN

Quite a few people have access to Gemini's 1M context length and it does seem to work very well.

E.g. can produce accurate scene-by-scene descriptions of hour long movies, can answer questions about very large codebases, etc.

This is a dream come true for me. Full movie descriptions! Or, uh, more likly for me, full video game playthrough descriptions!

Legitimately curious, what is lacking from current playthrough and movie descriptions available now? For games the only things I feel like might not be represented well in text is descriptions of the observer/player reactions and maybe details about their specific choices through a game... Kind of like turning a choose your own adventure book into a normal book by tracing one path through it?

I can see Q&A on a movie being useful but can't think of descriptions of the overall movie itself lacking... I'd definitely be worried about spoilers.

It may provide a way for AI to learn a complete new topic(a new programming language, stock investment, sport, games) in a zero shot way and immediately apply it by just feeding a youtube playlist/udemy course into an AI model.

For a blind person, yeah some games, like Mario and Zelda, and older Final Fantasy games, are hard to understand without descriptions.

The problem is mostly not summarisation but needles in the haystack search with a combined answer

Which Google claims Gemini is the best at so far

This analogy disregards all the work that happens behind the scenes to make modern Python workload work.

The current landscape is moving so fast that we don't stop and work on achieving the best version of a specific approach. One can argue that both RAG and having 1M+ context length is a symptom of a disconnect between what language models are capable of and what people think they can achieve with them.

Within the next 6 months there might be yet another approach to provide LLMs custom memory, and we will be having the same discussion the way things are handled currently.

This is 100% not true at all. Chips and electricity and the development resources would need to be enormous for your statement to be true.

It's really not even true for your python example.

Reply on news.ycombinator.com