I'm a Zionist. That is to say, I'm a Jewish nationalist, who immigrated to Israel about four years ago from personal conviction. As many will be aware, there is a strong current of Left thought that calls Zionism a racist ideology. This led to U.N. resolution 3379 (http://en.wikipedia.org/wiki/United_Nations_General_Assembly...) - later revoked as a condition of Israel agreeing to a crucial stage of the peace process.
But I would be very surprised if at least some of those who demanded that Yarvin be kicked for racism, did not also think that I was a racist on account of my Zionist beliefs; the phrase "progressive except Palestine" exists for a reason.
Now yes, Miller has every right to choose who comes to his conference, and the attendees have every right to ask that he kick people out for any reason they deem sufficient. But it seems to me that if there is a social norm of letting people veto speakers on grounds unconnected to the subject of their talks, then we're in a very parlous state - even if those grounds are things like racism.
Miller can't be blamed for uninviting Yarvin, there's too much money hanging on it. But this should not be something that passes without concerned comment.