Morally, I think people like Madoff are worse than many gang members or low-level violent criminals. That doesn't change the fact that they are far less of an immediate threat to the people around them, particularly now that they have been exposed. A confidence scam fraudster who has been exposed is a bit like a paraplegic murderer. They may be morally reprehensible, but they're effectively unlikely to reoffend in a few months awaiting trial.
If you honestly would rather live next to the average person arrested for assault or robbery than Bankman-Fried while both await trial, be my guest, but you can probably understand why others might feel differently.
Like many things, cash bail is effectively "discriminatory" in that it tends to hurt poor criminals more than white collar ones. That still doesn't mean discrimination is the primary reason it exists, or that that somehow negates any of its benefits.