I don't have a reasonable word to describe this (ie. nothing better than name calling). So let's just say "I think it's not particularly reasonable for a tech conference to put anything besides tech to be top priority" is an understatement for me.
That said, I want to note that any flak being shot should not be aimed at the organizer at all. The topic of "diversity" is slowly joining the rank of "What you can't say"[0] topic, and it takes a lot of work, courage and risk for an organizer to stand against an accusation like that. It's entirely reasonable for an organizer to drop a speaker under said circumstances, although they did botch the execution of removing a speaker pretty badly.
What's unfortunate is that such action can be considered "reasonable", and the question of who to send flak to is left as an exercise for the sub-comment thread.