I think it's because most tech jobs are not in tech. If you work for a bank, you work for bank. If you work for a fashion retailer, you work for a fashion retailer.

I've worked for a big pension fund and it was exactly like one imagines. Many meetings, a quiet office, slow and boring. I've also worked for small tech-focussed companies with technical nerds as founders; they were rather different, with music, dogs, vodka and late nights.

Oh and age matters too: after I had a kid, the late nights and vodka were not something I had the energy for.