By that singularity definition, we're probably nowhere near AGI, but if we define it as something that is as good at text/information manipulation as the 50th percentile human? I think we're already there.
By that singularity definition, we're probably nowhere near AGI, but if we define it as something that is as good at text/information manipulation as the 50th percentile human? I think we're already there.
> ...a hypothetical type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks.
You can argue that paperclip maximising is an inevitable consequence of that (and the huggingface breach is interesting from that point of view) but it's not fundamental to the definition.
The question then is what "surpasses human capabilities" means and we're there in some niches but not all, and not across many models.
But a purist defintion of AGI does not require consiousness as a part of it but can be seen as a benchmark metric. Sam Altman recently said the term doesn't really matter. And its true, even if AGI is achieved in the sense that a model can excel at all tasks at par or better than a human, then it is very useful but not as scary as a living, self-serving AI system like Skynet.
Yet AGI in the form discussed above will still grant massive power to AI labs if its injected into all domains. I recently wrote on this a bit on my blog: https://decodingvibes.com/blog/ai-can-ride-my-bike-with-no-h...
Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.
Also, if the lab truly has a self-improving superintelligence, the cost of retaining staff at any level would be a rounding error relative to its operating costs and the value the system creates.
There would be little economic pressure to fire them immediately, especially while they remain useful for oversight, interpretation, risk management or simply as "interface" to the rest of the world etc.
If anything, they would probably hire more people to pursue more opportunities in parallel.
This is why I keep saying we can't all agree even on a single letter of "A", "G", and "I".
Before ChatGPT, I would have said "obviously a generally intelligent system can do all the things". While LLMs are much more general than AI before them, the quality of their performance in all the things is distributed in a very un-human-like way.
Some fast-moving optimiser can be a threat well before it stops needing any humans for part of their labour. Cancer and viruses are examples of this: they're the same category of thing as a paperclip optimiser, but for biology instead of manufacturing office supplies.
But some others will argue LLM-spikey isn't "AGI", they'll demand something which reaches the performance of the best human (or the mean human, or the mean domain expert, because we can't agree on "I"), and a standard of "≥ best human" would mean that no, you don't need "people with knowledge around domain/layer the work is done on".
And yet… there are still people telling AI how to improve itself.
IMO there will always be a level of abstraction at which AI needs guidance. Perhaps ASI means it decides everything on its own, but I don’t think so. Genius humans often excel at the how but not the why, or even the what. So far there’s no indication that AI is different.
AI has not improved the network topology much yet. The next (and possibly 'last') big thing is enabling AI to come up with something as impactful as the transformer architecture.
I'm guessing that's hundreds of millions and maybe even billions of dollars per month in savings.
Honestly, when it comes to fundamental ANN topology improvement we've only just gotten started.