Curtis Yarvin is a bellwether - the sort of person that any group that starts excluding people for ideological disagreements, would probably exclude first precisely because his position is so problematic. So any group that accepts his technical contribution can obviously be trusted to tolerate any less-severe ideological disagreement. Conversely, any group that doesn't, especially when they have to make up nonexistent concerns to do it because their rules didn't cover this "obvious" reason to kick someone out and couldn't be hastily adjusted, must be viewed with caution.
I personally don't hold any beliefs nearly as objectionable as that. But I do hold objectionable beliefs - as I believe any halfway interesting person does. And those who don't, probably will eventually. Just stand by your convictions and give it time.
See also LambdaConf's conclusion on why they should allow him to participate anyway: https://degoes.net/articles/lambdaconf-inclusion I agree with this article fully.