Maybe a coincidence (I don't know details of computer history), but another interesting reason why binary is used might be information density.

Base e (2.71...) has highest information density. Of course, computers can't use irrational numbers as their base, so base 3 would be a closest pick. However, base 3 is much harder to represent in digital logic, so base 2 is a better pick and is still relatively close to e.

EDIT: Finally found a reference. https://en.wikipedia.org/wiki/Non-integer_representation#Bas...

I wonder if the idea originated before Hayes described it in 2001. Now it looks more likely to be a coincidence :)