That sounded awfully like you were accusing him of doing all this deliberately. But if you think it was all an accident, then okay.
Thought experiment: Your name is De Goes. You strongly believe that people's political opinions or personal characteristics should have no bearing on their professional life; to that point, you made sure to set up a blinded system for choosing LambdaConf speakers so nobody's prejudices, conscious or unconscious, could play a role. As a reward for actually practicing what Internet slacktivists merely preach, you have just discovered that one of the selected speakers turned out to be the infamous criminal Goldst -- er, Moldbug.
What's the PR move that gets you through this without fundamentally betraying your principles by giving in to the baying mob?