I think your opinion is valid to have. Mistakes are made that nobody would ever want to suffer from and regardless if the victims gain financial freedom from the mistakes (that rarely happens).
Realistically, I don't believe Judges will ever face real consequences and what's more probable is the system adapting towards lesser punishing modals. Also providing the ability to expunge records completely and universal income for criminals unable to obtain a job after their release. The punishment for a crime shouldn't be unemployment/homelessness for life.
There are other professions such as in the medical field that can really damage a persons life as well. Not everyone has the money to sue or even win a malpractice case and lawyers won't always take a victim's case as pro bono. Persons severely wronged, deeply need a way to feel some remedy and or we will continue to see vengeance outbursts. Especially from persons that see others being remedied for what they believe was much lesser in comparison to what happened in their own situation and where they constantly think the only reason they're not obtaining a remedy is from grossly unfairer situations they're in because of the wrong that happened in the past.
I think the foregoing is similar to why there are so many school shooters in USA. People need a remedy from being wronged and sadly all we have is a remedy system for the fairly well off. That also caters once in a while to someone that isn't so well off.