I would imagine that in a primitive environment, you will be eaten if you can't trust your hearing.
But I might be totally wrong, I haven't read the book.
E.g. clinical science does not pathologize people who claim to hear god during prayer and admit acting upon what they heard when the claims are in the context of organized religious practice and the behavior does not disturb the peace too much.
In my experience religion/spirituality felt very much like controlled schizophrenia (or my experience felt like an unconstrained version of it to be precise.)
Mental illness is pretty common but remains somewhat taboo. That's unfortunate.
When I talk to myself in my head, I normally don't find myself saying, "Who's there? Who said that?" Additionally, if someone was the physically appear in my room and start a conversation with me, I think I'd immediately figure it out. There's some mechanism that lets us differentiate from internal and external voice that we experience every day. I imagine the same applies on some level to auditory hallucinations, if those really are any different from having internal conversations.
That's not actually how schizophrenia works though; it's quite a bit more subtle/complex/varied than just the classic 'hearing things.'
Furthermore, I didn't say it was only that.