Hacker News Reader

Comment by tim333 | Hacker News Reader

On the same topic of AI helping scientific discovery there was this tweet yesterday

https://x.com/DeryaTR_/status/1972115494787338484

>...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections

>...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning...

Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?

View on HN

If and when these tools become self-improving we have absolutely no idea what comes next.

Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.

The architecture of the current models where learning is a separate and very expensive process make runaway self improvement seem like something that will require a bunch of breakthroughs in order to happen.

That seems reasonable, but I'm not sure if we really know yet. If current models are given direction and control to change how the next model is trained or architected it seems plausible that they could stumble into such a breakthrough.

The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.

I doubt there will be a sudden run away. Probably more of a gradual shift where humans and AI combine to improve the AI and gradually the AI does more of the work.

I'd agree that seems more likely. Though I can only make that expectation with public knowledge of what these AI companies say they're working on. I don't know what research projects or yet-to-be-released architectures may currently exist - my confidence in that assumption of likelihood isn't very high.

More importantly, I think we agree a sudden run away is possible, we don't have safeguards in place, and the risks are extremely high should that happen.

Reply on news.ycombinator.com