Is it because he isn't actually using the technology for work on a day-to-day basis like a lot of us?
I just cant get over this term, do you honestly believe in this? I use AI daily and while it is super useful I see too many limitations for it to “recursively improve and cause an intelligence explosion”.
1. Clearly, the people selling AI with this idea benefit greatly with this promise of infinite upside. Can you can trust them?
2. Singularity essentially requires to handwave away a lot of baked-in issues with LLMs or rely on unrealized innovation.
Counter argument: does anything else work this way? E.g. Moores law had an end too right? I would argue that the core tech breakthrough (Transformer-based LLM) has been improved, but no fundamental further innovation seems to have been made. The current architecture fundamentally hallucinates, even Fabel even on trivial problems. I.e. as number tokens increase error likelihood goes to infinity. How then, can this scale recursively to infinity?
If instead additional intelligence does little to speed up AI development (due to the need for other inputs like caputal and time), you could get a world where AI becomes better than humans at AI development and begins a cycle of recursive self-improvement without explosive growth leading to a singularity.
Which seems to me likely the result of an unforeseen variable or variables, and that's got to have outsized, uncharacterized, and unexpected importance to have such a strong effect.
When the overwhelming consensus is that wonderful things are waiting just around the corner, it still could turn out to be just the opposite and you'll never know until you actually turn the corner.
How is that not new?
Similarly, in my mind it can interpolate proofs by interpolating between data points for technique A and technique B. This is novel and brute-forcing proofs this way is useful. It is analogus to how sometimes it can generate programs that pass unit tests, I think.
However, creating fundamentally new concepts outside of the interpolated datapoints is not something I am convinced of. Maybe it can extrapolate some things, if correct add it as a data point, continue. Essentially a search, and it would be amazing if this works and maybe we can get some recursive improvement this way. But the “ideas” it will use to conduct this search are a function of the input data points as well, and thus in my view fundamentally limited in novelty. I am not discounting the usefulness, but I am not convinced you can just keep doing this indefinitely scaling intelligence exponentially.
Of course nobody can know yet really and I am just speculating just like you. But I also think the “experts” Sam and Dario also don’t know, and given their incentives I am not really convinced by them.
I have fears of dystopian worst-case scenarios, and dread the rapid pace of change and what it will do to real lives lived in the world, but it's only clanker cope/wishcasting that believes that (AI+human ingenuity) won't produce real AI ingenuity.
Re-read your comment in four years, I'll take the bet you'll find it very naive.
Well brute force is something I had waited for all my life so it could make a personal computing experience higher performance and more fulfilling.
By the time 1980 rolled around.
In some ways it's like a fundamental assumption that's always been there and nobody questions or really "thinks" about. It hurts the brain, sooner or later you want a computer to be able to do some thinking for you ;)
It's one of the easiest tech concepts to understand, especially for those who don't have very deep abilities.
Hardware is hardware, stronger is stronger, and quantity has its own qualities.
People in certain positions can go farther with nothing more than this firm a grasp of computer science or intelligence and it shows.
The only thing to be gained by continuing to grow the scale of "datacenters" from this point is the brutishness.
I expect it to pay off too, for those who can afford it, just not for everybody.
I think having a technical argument is important, as we live in a time with lots of hype merchants who stand to benefit from record breaking IPOs. Propaganda can affect us all, how do you know you aren’t being sold to?
Modern AI has taken those brakes off. Humans are taken out of the loop. IT infrastructure that modern society leans so heavily on, could be overhauled overnight into a very different beast. We wouldn't know what's happening before it hit us.
If anything, society should be reluctant to give AI powered systems 'hands & feet'. Use systems offline where possible. But we don't. Maybe some "Morris worm of the AI age" will get us to pay attention.
Whatever is coming, if it's bad we probably deserve it.