Hacker News Reader

Comment by bloomingkales | Hacker News Reader

I’m kinda getting the sense this is still just prompt engineering in a loop.

Persona-based prompting: We prompted the model to adopt different mathematical perspectives (e.g., "think like Euler focusing on series", "approach like Gauss looking for patterns").

I mean … I guess that’s scientific?

Besides that, how can the model learn at test time (at inferencing)?. It’s stateless, it doesn’t incorporate the last prompt into the model.

View on HN

It learns through the context. The context is state.

There's a bit of a cheat that's going on here though in that the model is being given the fundamental integration operations as part of the problem. That means the model hasn't had to learn what they are. It might not have needed to be given them, but it does feel like that's giving the model a leg up in the benchmarks that it wouldn't otherwise have, and when there's a direct comparison to (e.g.) DeepSeek, that's an unfair advantage.

Does the massive context still needs to be dragged around? Until get a neural network that adjusts weights in real time without relying on a huge clump of context being cycled there, I don't think there will be an AGI or even an impressive "agent". Current agents are just LLM looping lmao sold to people with no knowledge how they work at all.

I've been wondering about context compression, actually. I remember a bunch of prompt compression tricks from a couple of years ago that must be usable. If you were to say "anything past half context gets compressed, including the previously-compressed context" would mean you'd still have reasonable workspace and potentially infinite recollection of the most important things. Then yes, you'd be dragging a massive context around, but you're maximising the return on using it. I presume somebody's got a toolkit for this that I don't know the right terms to google for.

Compress first half (even recursively) is a nice idea, but you'd have to sort it by importance first – ie. you don't know if the most important sentence didn't exist at the beginning (ie. the whole goal could have been stated at the beginning, you don't want to loose it – you want memory to fade in terms of importance instead).

I don't know that, but the LLM might be able to make that judgement. If it's involved in the compression process, it can choose to drop cruft from the middle. Maybe "summarise then compress" is a better description.

Yep, let the LLM operate like functional programming functions. Explode(context) … map, reduce, flatten, explode, filter, find, filter, map, flatten, explode … reduce(context).

This is the imagination loop. OpenAI sold the prior reasoning loop. But that’s all this is.

Quite frankly, I wouldn't come to HN for anything less than this quality of content:

https://www.youtube.com/watch?v=DX3qLIwHoUo#t=1m29s

The machine reforming visually (explode, .., reduce) is what I'm describing.

There is https://www.rwkv.com/ which is an LLM based on RNN's, thus having "infinite" context length, it comes with its own tradebacks though. (Notably that its impossible to actually store infinite information in the network, so it prunes based on which information it finds more important.)

The "massive" context is still tiny compared with the size of the model whose weights you propose to update.

There's been some work on memory lately like Transformer² and Titans. But that may not be necessary for decent agents. Even the "context in a loop" is getting better as general reliability of tasks increases, and as people find better ways to manage context. Like Cline has the Memory Bank prompt where there's a folder of markdown files at diff levels of abstractions of info on the project, current feature, current task progress, etc. And so it can update those files and read from them as a consistent memory.

Reply on news.ycombinator.com