You can also put your cash into money market funds that invest in T-Bills, which is what anybody with some common sense and a large amount of money should do.
But in this world entities that offer you a return on your money will still exist, e.g. banks. If I can register my cash and you pay me x% per year, that’s better than getting 0. So banks would form regardless.
Banks make loans and create credit which allows the broader economy to grow. Better to have properly regulated private sector entities with skin in the game making lending decisions than having the federal government do it.
If you take a big chunk of credit creation and issuance out of service, the economy is likely to regress fairly significantly.
Clearly the existing bank regulations aren’t sufficient though.