Re. uncomfortable with finance: I guess it's a mostly "intuitive" ethical concern. I don't feel like finance is very much part of a society I'd like to live in or contribute to. I have heard arguments that e.g. trading is a big part of making the economy fluid and optimizing throughput or something, but it sort of rings wrong to me. Maybe it's just preconceptions and I should learn more about it!
Re. significant learning: Indeed, and that's part of the problem: since I'm not entirely sure where I want to go, I don't really have the mind to invest too much time learning about fields I probably won't work in, which makes my profile less attractive I guess. I originally had hopes that I could learn on the job: you often hear people tell you that a math PhD is proof that you can learn efficiently and have the capacity to work any technical field, or something. But I see now that it's probably not how it works: hirers still want some proof you're competent in their specific field.
Re. postdoc: the problem here is that the competition is rather fierce and the work conditions not always optimal.