A big part of it is exclusivity and branding.
I banked with SVB with my first startup. It got the job done, but the UI sucked, and there was nothing special about it compared to other banks. It was actually a very inconvenient bank to work with because SVB doesn't have any public branches so you can't walk into SVB and deposit a check or withdraw cash, it's all done over the phone, by mail, etc.
When starting my 2nd startup, I applied for an account and was denied. (What kind of bank denies a client an account?)
To be a SVB client, you need to be networked with other SVB clients (e.g. investors banking with SVB). This creates a big bubble where all of SVB clients are interconnected with one another, which might have been a good growth strategy for SVB by being "the exclusive" bank to be accepted into, but at the same time creates major concentration risk when all clients are in the same/similar industry (tech) and backed by the same group of investors.