Hacker News Reader

Comment by SubiculumCode | Hacker News Reader

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

View on HN

I hope this will happen, too. I think it might as soon as investors realize the LLMs will not become the AGI they were sold as an idea.

I think negative feedback loops of AIs trained on AI generated data might lead to a position where AI quality peaks and slides backwards.

I would not bet against synthetic data. AlphaZero is trained only on synthetic data and it's better than any human, and keeps getting better with more training compute. There is no negative feedback loop in the narrow cases we have tried previously. There may be trade-offs but on net we are going forward.

There's a pretty big difference between AlphaZero and a "generative AI" program: AlphaZero has access to an oracle that can tell it whether it's making valid moves and winning games.

By comparison, getting accurate feedback on whether facts are correct in a piece of text (for example) is much more difficult and expensive. At least, presumably that's why AI companies publish staged demo videos where the AI still makes factual errors half the time.

Automatic verification (oracle) is being used today to create synthetic data for LLMs. I don't see it as a big difference versus AlphaZero. While there's no way to ensure that a single synthetic reasoning trace is correct, as long as it leads to the correct answer according to the verifier, the law of large numbers should take care of that.

The problem is that it's difficult to create verifiers for many things we care about like architectural taste. So I expect to see superhuman capabilities on the things we can make verifiers for, but for other things it's harder to predict. We may see transfer learning or we may see collapse. My money would be more on transfer learning.

Transfer learning is one of the biggest unsolved problems in AI. And we are nowhere near solving it or even understanding how to go about it from an algorithmic perspective. We will definitely see collapse of the current hype train before we understand and employ effective transfer learning.

AI will radically leap forward in specialized function gain over the next decade. That's what everybody should be focusing on. It'll rapidly splinter and acquire dominance over the vast minutia. The intricacy of the endeavor will be led by the AI itself, as it'll fly-wheel itself on becoming an expert at every little thing far faster than we can. We're just seeding that possibility now. Not only will it not slide backwards, it'll leap a great distance forward from where it's at now.

Mainframes -> desktop computers -> a computer in every hand

Obese LLMs you visit -> agents riding with you whereever you are, integrated into your life and things -> everything everywhere, max specialization and distribution into every crevice, dominance over most tasks whether you're there or active or not

They haven't even really started working together yet. They're still largely living in sandboxes. We're barely out of the first inning. Pick a field and it's likely hardly even at the first pitch for most of them you can name, eg aircraft/flight.

In hindsight people will (jokingly?) wonder whether AI self-selected software development as one of its first conquests, as the ultimate foot in the door so it could pursue dominion over everything else (of course it had to happen in that progression; it'll prompt some chicken or the egg debates 30-50 years out).

Ah the old "everything I'm into is the model-t/iphone" which is why I'm programming in my metaverse home on the block chain.

Thank goodness we have version control systems then.

"Version control systems", in case of AI, mean that their knowledge will stay frozen in time, and so their usefulness will diminish. You need fresh data to train AI systems on, and since contemporary data is contaminated with generative AI, it will inevitably lead to inbreeding and eventual model collapse.

We are just at the beginning of integrating external tools in the process and developing complex cognitive structures. LLM is just one part of it. Till now it was cheaper and easier to improve that part especially if other work would be rendered obsolete by LLM improvements.

Please show the evidence of more productive. How did you measure it?

The amount of human suffering and death that could be massively mitigated by advanced AI is overwhelmingly worth the unknown risk in my opinion. If you had people close to you die from something where medicine or healthcare resources are close but not quite there to have allowed them to survive then you might feel the same.

I hate this argument, because all you have to do is look around the world today to see that if we have massively powerful technology that is controlled only by a few that it sure ain't leading to the "think of all the diseases we can cure!" utopia you describe.

We have many, many people around the world die all the time from easily curable and preventable diseases, we just choose not to. This is largely not a technology problem. Just look at PEPFAR, which saved tens of millions of lives from HIV/AIDS. We just decided to stop funding it: https://en.wikipedia.org/wiki/President%27s_Emergency_Plan_f...

Reply on news.ycombinator.com