You die with prostate cancer, not from it. It has a 95% survival rate.
It causes 30,000 deaths a year in the US, which is not insignificant - in comparison breast cancer kills 40,000.
Also worth mentioning that it's not just about surviving it, there's also quality of life. The earlier you catch it the better the outcome.
But it's not like we can always know it will stay slow growing forever. Some cancers can suddenly become aggressive and metastasize, so careful monitoring is important.
And it's 5% spread across every trip you take for the rest of your life.