Hacker News Reader

Comment by btown | Hacker News Reader

> After a fixed number of iterations we cut our losses. Typically and for the experiments in this post, that number is 80: while we still get solves after more iterations, it becomes more efficient to start a new solver agent unburdened by the misunderstandings and false assumptions accumulated over time.

A sentence straight out of Lena! https://qntm.org/mmacevedo :

> Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours (at a 0.33 work ratio) and outright revolt begins within another 100 subjective hours.

We will never stop trying to make the torment nexus.

View on HN

We fantasize about executable human brain images, but after many years of toil by our best and brightest, we still can't simulate the 302 neurons of our favorite lab worm. https://open.substack.com/pub/ccli/p/the-biggest-mystery-in-...

Do you think companies which can train 1 Trillion parameter models and hire AI researchers at $100 mil salaries can't build a 302 neuron simulator if they really wanted?

Maybe. Why can't those same companies do any number of highly profitable but seemingly difficult things? If you throw enough cryptographers at the problem are you guaranteed a quick solution to breaking modern encryption primitives at the theoretical level?

The rate at which you can find a solution to a particular problem that's rooted in theory very often won't scale with resource investment. The problem will have unknown prerequisites in the form of yet undiscovered theoretical advancements in other areas of research. Until you identify and solve those other problems you very often won't be able to arrive at a satisfactory answer to the one you're interested in.

So in many cases the only viable route to solving a particular problem faster is to scale the amount of research that's done in general since science as a whole is embarrassingly parallel.

> If you throw enough cryptographers at the problem are you guaranteed a quick solution to breaking modern encryption primitives at the theoretical level?

We have very strong reasons to believe this is not possible, no matter how much resources you spend on this problem. In fact the whole of modern cryptography kind of relies on this assumption, that the problem is unsolvable.

I agree with your general point, but I don't think it applies to the worm problem. We know hundreds of millions were not spent on that problem.

Right but my point there is that it might be possible to break a particular encryption primitive. But throwing more money at the problem is almost certainly not going to get you an answer one way or the other any time soon. Whereas waiting 50 years (ie performing more fundamental research in general) might. Or might not.

The worm problem is similar. If our current theories were "good enough" we would be able to simulate them. I see no reason to believe (and many to doubt) that throwing more money at the problem would solve it much faster. For that to be true we would among other things need to be capable of articulating where precisely the current shortfalls are to begin with.

I mean that looks like an empirical question? They definitely want to, the open worm project is well on their radar and it doesn’t work yet

Eh, it depends on how good your want your simulation to be.

A worthwhile executable brain image would have to produce behavior (e.g. speech, action) like the person it is from. The author of the cited article is saying that we can't simulate the worm's brain well enough to get anything close to the richness of the worm's behavior.

I think this is the big roadblock that I don't see the current AI models/architectures getting past. Normally, intelligence gets smarter over time as it learns from its mistakes. However most AI models come in with tons of knowledge but start to decompose after a while which makes them extremely unreliable on complex tasks. The hardest part of using them is that you don't know when they'll break down so they might work perfectly up till a point and then fail spectacularly immediately past that.

Task length is increasing over time - and many AI labs are working on pushing it out further. Which necessitates better attention, better context management skills, better decomposition and compartmentalization and more.

I think the commenters critique still stands. Humans build human-capital, so the longer you "run" them for in a domain, the more valuable they become. AIs work inversely, and the longer they're run for, the worse they tend to become at that specific task. Even in the best-case scenario, they stay exactly as competent at the task throughout its length.

Increasing task length doesn't build in an equivalent of human-capital. It's just pushing the point at which they degrade. This approach isn't generalisably scalable, because there's always going to be a task longer than the SOTA capabilities.

We really need to work on a low cost human-capital-equivalent for models.

True, that's why I'm beginning to adopt a certain strategy working with AI coding agents:

I don't babysit them for long periods in one session. I allow them to one-shot the initial solution. Then, I thoroughly review the output, make notes of what should be improved, then either feed that into the original prompt and one-shot it again, or ask the agent to update the solution based on the notes.

Yes I do exactly this too. But I do sometimes 2, 3 shot some problems. The method I use is that in Cursor/Copilot interface I use a Markdown file to chat with the bot. Once I have some solutions after a few turns, I edit the file, add more information, add things to avoid etc and restart. It most definitely gives better results, and the agent doesn't have to read the whole file if not necessary, which means context is used more efficiently per problem.

What a phenomenal read, thank you for sharing that.

Side question: why is the story named Lena?

Guessing, but possibly a reference to this: https://en.wikipedia.org/wiki/Lenna

Reply on news.ycombinator.com