However, each PIN entry is accompanied by "code check": bank's support person says their 4 digit code, and you can verify that it matches on request for PIN screen. This neatly prevents someone pretending to be a bank during a call, because each PIN request uses a different "code check".
Attacker wants Victim's code. Attacker calls the Bank impersonating Victim, and also calls Victim impersonating the Bank. Bank tells Attacker the code check, Attacker tells Victim the code check, Victim sees the match and enters their PIN into the Smart-ID app, and Attacker's phone session with Bank is now fully authenticated and has no more need for Victim.
WebAuthn is really what is good enough. Luckily it's well supported on all important platforms so there's really no excuse using anything worse.