the richest people i know (with wealth in the 10M range) are the most amoral in their thoughts and actions, and it makes a big difference financially. note that i said amoral, not immoral; they aren't malicious, they simply don't care.
they don't refuse a business opportunity because the business would be explicitly unethical; they do it, and profit from it. the same is true if the business produces negative externalities. it simply doesn't matter to them because money is money as far as they care.
they don't craft their investing strategy with the goal of investing in companies that behave well; they chase the money regardless of the ethical problems that they may be directly incentivizing.
overall, this gives these people a much greater range of freedom when it comes to making large volumes of additional money. while i don't know for sure if being amoral helps people become rich in the first place, i assume that it does because having more options on the table is never a bad thing.
so, maybe the answer to the question of "if you're so smart, why aren't you rich" is "i refuse to act unethically to make money".
in my personal experience, i think this argument is pretty good. in 2010 or so, i realized that there was a then-latent market for TDCS / consumer nootropic brain stimulation systems, and considered making a company around producing and selling them to consumers.
however, at the time, i knew that brain stimulation hadn't been proven safe to use in the long term, and selling it to consumers would require working around the FDA's rules about medical devices to essentially evade their oversight. in short, i refused to start a company whose time had come because i knew that i'd probably be harming people as part of the core business. i would have been one of the first in the space, and while i don't think i would be rich today if i had done it, i know that it would have been a chance to get rich and certainly that there were some people who did get rich. and of course, there have been plenty of consumers who have been harmed by these products, often in unidentifiable or subtle ways.
so what's the moral of this story? having morals is expensive, and it sometimes precludes you from becoming rich even if you're lucky enough to be at the right time to take action.