Sure. If you apply our standards, though, you might want to decry many of your founding fathers as well: Washington, Madison, Jefferson ("All men are created equal"), John Hancock, Benjamin Franklin - they all held slaves at some point.
So, should you rename the capital, the state, and all the avenues and parks that are named after these people?