You are not wrong at all. There is a certain level of fraud tolerated by banks so that they can more easily make loans to people. It's the classic security vs. convenience trade-off.

Two big problems are:

1. If you happen to be one of the victims of the fraud, it hurts! Sometimes a lot! A lot more than it hurts the bank.

2. If you don't like the level of (in)security that the banks have chosen, what other options do you have? Right now I don't know, I think maybe Bitcoin is your best bet?