The reason why true singularity-level ASI will never happen isn't because the technology cannot support it (though, to be clear: It can't). Its because no one actually wants it. The capital markets will take care of the rest.
Plenty of people want something capable of doing tasks well beyond what GPT-5 can and that are equally capable to a proficient human.
If you can do that cheaper or faster or more available than said skilled human, there is definitely a market for it.
As much as we hate on meta, open models are the answer.
My point in being pedantic about this is to just point out that an extreme amount of value could be generated by these systems if they only attained the capability to do the things 110 IQ humans already do today. Instead of optimizing toward that goal, the frontier labs seem obsessed with optimizing toward some other weird goal of hyper-intelligence that has very unclear marketability. When pressed on this, leaders in these companies will say "its transferable"; that hyper-intellectuality will suddenly unlock realistic 110 IQ use-cases; but we're increasingly seeing, quite clearly to me, that this isn't the case. And there's very good first principal reasons why you should believe that it won't be.
But 180 IQ intelligence that doesn’t sleep doesn’t stop with instant access to the worlds knowledge will absolutely be able to revolutionise the sciences.
But something that has equivalence to high human intelligence (say 140IQ, generalized to a wide variety of domains) and can engage in long-term directed, purposeful, and autonomous behavior will (not all humans of high intelligence cross this bar, but the ones that do tend to make an impact).
And sure, it's not perfectly transferable. It's going to be a long time before it can fix my plumbing, but there's a fair number of tasks that it can do now that are useful.
> toward some other weird goal of hyper-intelligence
I think the near term goal is something that can do most human tasks of the scope of tens of minutes of work at a computer, without task selection or self-direction. In some ways this looks superhuman, doing this stuff at high speed and 24/7. But it's superhuman like a mechanical loom is superhuman.
Even better would be something you could delegate higher level goals, including tasks of larger scope and duration, but that's slower coming.
...Why would it not be intellect?
What we're missing is anything that actually resembles thought at anything more than the very surface level.
LLMs are not intelligent. They sound intelligent because they're being trained very well at predicting what tokens to produce to sound that way, but they have no intellect, no understanding, no world model, no consciousness—no intelligence.
I'm eagerly waiting for the next big breakthrough when someone trains a model that switches between "thinking" and "outputting" modes mid-answer :)
??? That was last year's breakthrough.
Generating tokens to represent steps isn't just "cosplaying" --- it's representing more state than can fit into a single step of the model.
Ditto for when I talk to myself when figuring something out, either out loud or just mentally.
> Generating tokens to represent steps isn't just "cosplaying" --- it's representing more state than can fit into a single step of the model.
Well duh, obviously, but once you look closer, some of these "steps" aren't really any actual steps - they're just cosplaying. You can often catch a model outputting something like "I'm collecting the list of resources to base this table on", even though it doesn't do anything close to that (which is hopefully obvious). It just writes this, because it's a reasonable thing for someone who researches a topic to include in their stream of conscio... I mean, chain of thought. This then also helps it move the weights for future tokens in the direction of a writing from someone who appears to have collected such list, the same way people had more success with early image generation by adding "award-winning" or "high quality" to the prompt. Remember "prompt hacking"? Not in the security sense, but in the "I'm a prompt engineer" sense. The models prompt-hack themselves now. These breakthroughs consist of obvious next steps, it's just that someone with deep enough pockets allowed them to be tried on big enough datasets with long enough compute.
Except it seems like it's a worse google, therapist, life coach and programming mate, with the personality of someone who spends all their time trying to solve the Riemann hypothesis.
But the rest I agree with.
It's just a different tool than Google, and quite complimentary in many ways: instead of doing many keywords searches to collect information, aggregate it in a mental model, and derive a conclusion, it's easier to ask it to answer the aggregation I need and back-verify if it makes sense. To me it has helped a lot to not have to know the right incantations of keywords for the information I'm trying to find right out of the bat, and from the LLM answer I can backtrace what searches to do to confirm the validity of it.
It's a small twist but it has definitely showed me some value in these tools for more general information lookup.
The thing is: I still want and need to have a search engine, if for some reason search engines cease to exist and LLMs take their place I will not be able to trust anything...
ASI isn't what people want or not. It's an AGI that is able to self improve at a rapid rate. We haven't built something that can self improve continuously yet. It's not related to whether people want it or not. Personally, I do want ASI.
> We haven't built something that can self improve continuously yet.
Actually, we have. Its called AlphaGo; its capable of rapid self-improvement on the intelligence vertical of playing Go. Is that ASI?
If your answer to that is No, then you've accepted the base case of a more general idea: That the vertical of self-improvement does matter in the human designation of ASI, which means the objective and reward functions defining "ASI" are at some level human designated.
In other words, it is deeply related to what humans want.