> Our mental powers, it is argued, must outstrip those of any computer, since a computer is just a logical system running on hardware, and our minds can arrive at truths that are beyond the reach of a logical system.
For a much more intuitive explanation as to why consciousness is not algorithmic, I recommend "The Neural Basis of Free Will" by Tse. His argument is that neurons and neuronal circuits (and more) harness randomness to provide inputs to "criterial detectors" which are satisfied when the right combinations of inputs (spatiotemporal patterns) arrive at the detector at the right time. This can't be algorithmic, because of the requisite noise in the inputs and because the brain realizes true parallel processing. As a further note, he posits that free will is realized in the resetting of the input weights, so "current" actions set up the criteria for future actions avoiding the issue of causa sui in free will.
Perhaps this is a Turing test for consciousness. "I can't prove this, but I've been thinking about these theorems [inserts true but unprovable list of theorems] and I think they're true".
A test requires that you be able to produce an artifact which is witness to the results -- what could you produce as the result of the test to distinguish between a true, but unprovable theorem and a nearly true theorem with a single (very, very, VERY) large counter-example?
A "true but unprovable" Goedel statement is true in standard models of arithmetic but untrue in certain nonstandard models. The "incompleteness" is syntactic, not semantic. The real and complex numbers, AFAIK, only have one model, up to isomorphism.
And sometimes statements are difficult to prove because they're actually independent of the foundational system. Or because a counterexample exists somewhere.
"This problem is unresolved, therefore it's a Goedel Statement within our current foundational system" is extraordinarily unlikely. For one thing, that would imply that we could figure out the axiom we're missing and pass to the stronger system capable of resolving the conjecture straightaway, or that starting from some stronger foundation like homotopy type theory would resolve the conjecture right-away. Most unresolved conjectures are not unresolved for lack of proof-theoretic strength in our foundations.
A simulation of a brain or some set of neurons is an algorithm, as are algorithms that use nondeterminism or that simulate parallelism.
A lazily evaluated, natively parallelized stochastic lambda calculus with Church-like query operators is still merely probabilistic-Turing-complete.
Arrrrrgh now you've got my CS401 knowledge screaming that someone is wrong on the internet.
You could write a computer program to generate random propositions, and test those logically against known truths. After enough testing, the program could assert that propositions not disproven are true. Voila! You have a computer program that has "arrived" at truths outside of a logical system. Some of those truths will be wrong, and some will be right, much like humans and their "truths" that are not based on logical reasoning.
If you've never encountered a human claim to "see" the truth of something that turns out to be false, well, it happens... a lot.