Godel himself had his quirky beliefs about the topic which Penrose seems to just be transmitting.
Godel believed that humans had a trans-computational understanding because people could see through the incompleteness theorem but a computer cannot. Hence people have some transcendental (for lack of a better word) cognitive grasp.
I think Heidegger is a better source to draw from, and who also is a philosopher with a proven track record for influencing AI substantially (through Dreyfus, Winograd, et al). These models have no true being, they don't care about anything, they don't wake up and aim towards anything. They have no true embodied, embedded, purposeful existence. This is really what Penrose means by being "conscious."