Call me crazy, but I believe that you don't have to actively champion and invite openly racist people to conferences to show that you tolerate difference of opinion.
If you're protecting personnel, even after a number of others in your community have shown disagreement with the person's actions (and protections afterwards), just admit you agree with those thoughts. If not, your entire organization is cowardly and hiding behind a scapegoat and mouthpiece.