Another good line to look at is how people believe in ghosts: people with established religions without "ghosts" are less likely to believe in ghosts than people with atheism, even when they'd supposedly be skeptics superstitious claims.
Having functional paradigms is important, and being confident that there isn't a magical extrapolation into AGI is healthier than there being some magical exponential increase that you have to ride the dragon.
Sorry man, we're not solipsistic here. There are reasonable beliefs that are justifiable, instructive and then there are ones that require cherry picking technology indistinguishable from magic without reference to reality and physics.