You could apply your whole argument to public opinion and referendums - public will change mind a lot, why make such drastic moves based on it?
It was “states that have slavery get to count 3/5 of their enslaved population when determining the representation the free population gets in Congress (and, hence, also it's voting power in Presidential elections.)”
It's actually worse than black people counting as 3/5 of a white person; its more like slaveholders counting as 1 + (slaves held × 3) / 5 people.
On the other hand, public supported all those "bad" laws at the time ... But yeah, I was wrong, illegality on its own is wrong argument.
To suggest that, e.g. Gandhi didn’t have a meaningful role in achieving India’s independence, because the British considered his actions illegal, is very silly.