I don't agree with all of the OP's arguments, but wow, what a great little piece of writing!
As the OP points out, the "accelerators vs doomers" debate in AI has more than a few similarities with the medieval debates about the nature of angels.
As the OP points out, the "accelerators vs doomers" debate in AI has more than a few similarities with the medieval debates about the nature of angels.
If you like this essay from The Economist, note that this is the standard level of quality for that magazine (or, as they call themselves for historical reasons, "newspaper"). I've been a subscriber since 1985.
"Surely I come quickly. Amen."
Currently humans are still in the loop, but we already have AI enabling advancements in their own functioning at a very primitive level. Extrapolating from previous growth is a form of belief without evidence since past performance not indicative of future results. But that’s generally true of all prognostication and I’m not sure what kind of evidence you’d be looking for aside from past performance.
The doubters are dismissed as naive thinking that something is outside our ability to achieve something, but that’s only if you keep moving goalposts and treat it like Zeno’s paradox. Like yes, there are weaknesses to our current techniques. At the same time we’ve also demonstrated an uncanny ability to step around them and reach new heights. For example, our ability to beat Go took less time than it took to develop techniques to beat humans at chess. Automation now outcompetes humans at many many things that seemed impossible before. Techniques / solutions will also be combined to solve even harder problems (eg now LLMs are being researched to take over executive command control operations of robots for example instead of using classical control systems algorithms that were hand built and hand tuned)
Things which used to be considered AI before we solved them, e.g. automated optimisation of things like code compilation or CPU layouts, have improved our capacity to automate design and testing of what is now called AI.
Could stop at any point. I'll be very surprised if someone makes a CPU with more than one transistor per atom.
But even if development stops right now, our qualification systems haven't caught up (and IMO can't catch up) with LLMs. Might need to replace them with mandatory 5 years internships to get people beyond what is now the "junior" stage in many professions — junior being approximately the level which the better existing LLMs can respond at.
"Transform society" covers a lot more than anyone's idea of the singularity.
From the same entry:
> However, evidence that the question was widely debated in medieval scholarship is lacking.[5] One theory is that it is an early modern fabrication,[a] used to discredit scholastic philosophy at a time when it still played a significant role in university education.
Clearly a lot of "effective altruists" for example, want to argue that the most altruistic thing they could possibly be doing is to earn as much money as they possibly can and horde as much wealth as they possibly can, so they'll come up with a tower of logic based on far-fetched ideas like making humans an interplanetary species or hyperintelligent AIs, or life extension or whatever so they can come up with absurd arguments like: If we don't end up an interplanetary species, billions and billions billions of people will never be born so that's obviously the most important thing anybody could ever be working on, so who cares about that kid starving in Africa right now. He's not going to be a rocket scientist, what good is he?
One thing most philosophers learned at some point is that you need to temper rationalism with a lot of humility because every chain of logic has all kinds of places where it could be wrong, and any of those being wrong is catastrophic to the outcome of your logic.
This mechanism is exactly why the more intelligent a person is, the more likely they are to believe very weird things. They can more easily assemble a chain of rational logic to lead them to whatever it is that they want to believe.
Very much a double edged sword.
Fuck em.