If the company doesn't get off the ground (vast majority of investments) you lose all your money.
If the company does get off the ground, you are the lowest on the pref stack, and you have no ability to follow on to protect your position. You're not a contributing employee or meaningful future source of capital so your piece of the pie is just dead weight on the cap table. This means every single subsequent investor (and the founders, if they care more about money than their relationship with you) has an incentive to cram you down.
So net net the chances of success from passive angel investing are only slightly better than playing the lottery.
Best approach would be to make very few investments, where you're able to build a special relationship with the founder, and ideally get a board seat to defend your stake.
===
Edit - to be clear, I don't think startups should be giving board seats to angel investors. It does happen in exceptional cases where the angel is uniquely valuable to the company, and those are the cases where the angel can defend themselves. But they are rare, which is why it's mainly a bad game to play.