The ACLU is also a liberal association[1], although I agree liberals and conservatives work together in some issues. There's no doubt about it.
And yes, I think it's okay to reject them, because I am afraid of them gaining affection from everybody just to push an agenda later. If they wanted to herald digital freedom, they should have remained completely neutral in political issues, but they chose not to...
[1] https://en.wikipedia.org/wiki/American_Civil_Liberties_Union...