So unless you pose that a function has to rely on its materialization (there is something untouchably magic about biological neural networks, and intelligence is not multiple realizable), it should be possible to functionally model intelligence. Nature shows the way.
AGI will likely obsolete humanity. Either depricate it, or consume it (make us part of the Borg collective). Heck, even a relatively dumb autonomous atom bomb or computer virus may be enough to wipe humanity from the face of the earth.
Even if we assume for the sake of argument that AGI is possible, there's no scientific basis to assume that will make humanity obsolete. For all we know there could be fundamental limits on cognition. A hypothetical AGI might be no smarter than humans, or might be unable to leverage it's intelligence in ways that impact us.
Nuclear weapons and malware can cause damage but there's no conceivable scenario where they actually make us extinct.
I agree our knowledge currently is lacking, but see no reasons why this will never catch up.
There are fundamental limits on cognition. For one our universe is limited by the amount of computing energy available. Plenty of problems can be fully solved, to where it does not matter if you are increasingly more intelligent (beyond a certain point, two AGI's will always draw at chess). Another limit is practical: the AGI needs to communicate with humans (if we manage to keep control of it), so it may need to dumb down so we can understand it.
Even an AGI as smart as the smartest human will greatly outrun us: it can duplicate and focus on many things in parallel. Then the improved bandwith between AGI's will do the rest (humans are stuck with letters and formulas and coffee breaks).
Manually deployed atom bombs and malware can already wreck us. No difference with autonomous (cyber)weapons.
And what does alarmist even mean? Do you call global warming advocates alarmists? It’s such an annoying, nonsense word that boils down to name-calling really. Discuss the merits of my actual argument. If you think my speculation is wrong, point out a flaw in the chain of logic that leads to my conclusion. Don’t just wave your hand and say that “you can’t prove it” like some evangelical christian talking about god or global warming. Seriously infuriating when there is so much at stake.
The analogy to anthropomorphic global climate change is a non sequitur. Climatologists have created falsifiable theories which make testable predictions.
And you really have no clue about my personal religious beliefs. Calm down and take a seat.
I would argue that, unless you can show why AGI is not - in principle - possible, that the null hypothesis would be that it is possible. Unless we veer off into some weird mysticism, it seems that the human brain turns energy and matter into intelligence somehow, operating according to the physical laws of the universe... why shouldn't it be possible to build something else that does the same?
If you're unwilling to provide me with a prototype equivalent to a rodent mind then I'll settle for a fully developed theory of human cognition. Let me know when you've got one. At least that would give researchers some guidelines to know whether they're making forward progress toward AGI.
Yeah and before the measurements were done, before enough time had elapsed for meaningful change to be measured, all there was was people like me screaming at people like you, trying to make you see. When ai comes you’ll have your proof but it will be too late.