It's easier to argue it's not GPT-3 that's advanced, but it's humans that are simple.
Brains are structures in the physical world and you can model their behavior with math. ("Model" is not the same as "perfectly model." Imagine a three-body problem where you can come up with plausible solutions. That's not the same as an exact solution.)
What happens when you multiply 2 times 5? You got 10? That's the same answer I got!
Thoughts are executed by a brain, which can be modeled and executed by a mechanical brain.
Many of the arguments made by GPT sceptics are alarmingly close to the intelligent design argument against evolution. If a really dumb process with countless billions of iterations can produce human beings, why couldn't a different dumb process eventually produce human-level machine intelligence? Until the curve of progress shows some signs of flattening, I'm not going to speculate about what GPT-4 or GPT-5 will or won't be able to do.
(and just as sceptic arguments can resemble ID and dualism, the pro-AGI argument that intelligence selectively guiding an evolutionary process is as plausible a route to consciousness as random chance and a sufficiently large quantity of atoms and time has similar superficial parallels with arguments for 'theistic evolution', not to mention near-future curve exponentiation arguments resembling millenialism)