I'm surprised that the HN community seems more averse to using FB connect than I expect most normal users are. Either that or people that are critical of FB Connect are more apt to respond to this.
You're probably seeing both effects here. As a moderately tech-savvy person, I've got just enough information to be dangerous -- I know that there are/have been missteps with facebook sharing information (and there was that business with sites automatically publishing purchases to your feed) but I don't know enough to know exactly what is/isn't safe. So I err on the side of caution. Someone who hasn't seen the headlines in the tech press might not be as cautious.