Counter point: I was able to convince Gemini that a human baby is larger than a Saturn V rocket by simply telling it as much, to which it responded that its previous statements had been wrong and I was indeed correct.

The models being sycophantic and suggestible is another issue, but it’s not hard to get them to agree with false information.