Users get scared when a site asks them to login via Facebook or even Google. If this is the only option you offer, registrations will suffer a lot. I don't know who would use this.
I'm curious - do you have any data supporting this? I see it on HN based on anecdotal evidence (or none) all the time. It would obviously have different effects on different market segments, but I would love to see some analysis on this.
I'll provide my personal input, if a website requires a Facebook account to log in, I won't do it. I have the Facebook Apps platform turned off for a reason.
No hard data, just lots of complaints when I tried it.
I'm very unlikely to sign up for a new site if it doesn't allow login with My Google account via OpenID. I'm sick of making up new passwords and keeping track of them.
For some mass-market sites, scaring off those paranoid, nitpicky, self-righteous users could be a winning move.