Without finance, there is no biomedical research (private especially). Without finance, there is no aerospace advancement. Without finance, there is no alternative energy. Being able to facilitate a more liquid, more transparent financial sector is, in my opinion, a calling worthy of any programmer who seeks to make the world a better place.
Your assignment of people as 'questionable' is likely an artifact of the 2007 recession, and that kind of thing is a personal choice. I can guarantee you that there are 'questionable' people in every sector and in virtually every company. Sometimes people are great, and sometimes they aren't. Their field of employment has very little to do with what makes them that way, and I think that it's a little ignorant to be so casual (and I say this because you are certainly not alone in your judgment) with how you view people.
Remember how, as children, we are taught to not judge books by their covers? Why do the same with a programmer who works in finance? Sure, some of the work may be a little pedantic, like squeezing an extra millisecond out of an algorithm, but hey, that's computer science in general, and I liken that to an F1 racing team spending hour upon hour sculpting the perfect frame for their car.
Sorry for the rant, I've spent a decent amount of time working on financial algorithms, and what you said touched a nerve.
Have a great day!