You're right. I knew language is not necessary for cognition, but I always thought of thought as just internal monologue, i.e. "trains of thought". It turns out the definition they use in cognitive science is "the manipulation of internal mental representations to guide behavior, solve problems, and model the world in the absence of direct sensory input". Which doesn't require language either.
Probably it's this linguistic component, together withs shared intentionality/cooperation, that makes all of the difference when it comes to the power of human cognition, so I wouldn't say animals merely lack this component, but it's true that they appear to have pretty much everything else we do.