Malware on your phone can reroute your calls to the attacker. So you think you're calling the official number at the correct institution, but you're actually talking to the attacker.
If some malware is that deep on the phone, able to redirect calls, then you've got much bigger problems and the attacker might not even need to trick any cooperation at all.