Probably tough to fix small models with fine-tuning... You'd likely need to train them with enough data from languages (and maybe translations between them) that they need to support, so that they can connect ideas in different languages properly.

Separately, the fact that they can translate between languages where they have never seen any translation data in their inputs shows that they have internal models of the world and language, that go beyond what one might expect from a statistical parrot.

They do have world understanding - perhaps limited some by the fact that their input data may not cover a lot of everyday things.