I've got an account number, which is only used by the bank (though it appears on all bank statements). For some operations (adding a beneficiary for bank transfer, for example), there's a double factor, either on my phone or by postal mail and then there's at least a 24h delay: they must know their security is bad, so the bank limits the damage that can be done, you can't just get hacked and all your account emptied in a hour.
And to change your phone number, you've to use a on-use password, sent by postal mail to your address.