I straight-up don't believe this. Can you link to the claim so I can understand?
Surely if "high-order thought" has any meaning it is defined by some form. Otherwise it's just perception and not "thought" at all.
FWIW, I don't "hear" my thoughts at all, but it's no less linguistic. I can put a lot more effort into thinking and imagine what it would be like to hear it, but using an sensory analogy fundamentally seems like a bad way to describe thinking if we want to figure out what it thinking is.
I of course have non-linguistic ways of evaluating stuff, but I wouldn't call that the same as thinking, nor a sufficient replacement for more advanced tools like engaging in logical reasoning. I don't think logical reasoning is even a meaningful concept without language—perhaps there's some other way you can identify contradictions, but that's at best a parallel tool to logical reasoning, which is itself a formal language.