The big investment banks are struggling to be profitable these days and it has nothing to do with Brexit or the EU. After the financial crisis in 2007 they have had to hold more and more capital and be more transparent about their complex transactions. And rightly so. In order to be competitive they build software that lets them comply with the regulations as tightly as possible, to maximize profit. The more the financial regulators see the more they regulate. Smaller institutions like hedge funds don't bare such a high burden. My building has been quietly emptying out for two years and so are the others. The giants are dying but it's not because of Brexit. I see purely software companies and specialist hedge funds taking over in this area in the future.
Truth be told, Brexit will hurt the EU more than it will hurt the UK and I won't hold any grudges if the EU member states fought hard to defend their union. If they impose punishing import and export duties on us then it will most likely not be out of spite but to protect them from other member states leaving too. Picture yourself on the other side of the fence, what would you do?
And one last thing, the EU was created to economically tie as many countries in Europe together so they would all depend on each other for resources. That way no one country could start world war three. By design it should maintain connections and dependencies regardless of what localized people living in it desire in the short term. Making member states more wealthy and comfortable is secondary.