This could be a good end goal of a startup, but it can't be what they do initially.
In the first case you'd have to start with a tiny, specialized payments market that you could then expand. The second case would be Facebook creating their own credit card system. Which they could probably do.
I'd recommend starting with a specialized payments market. That's how credit cards themselves got started. And you can't create Facebooks for the asking.
Now, the difference between Square and the CC companies is that square is really a gateway to those CC systems and not a credit provider themsleves... BUT once they have traction with POS merchants and acceptance by consumers could they start their own credit department? Sure. Would they want to? That is another debate. I'd wager it's much easier and still very profitable to be a gateway taking a small slice off each transaction than it is to be the credit provider.
I know the guys who put Dwolla together. It shows a lot of promise, and is a huge boon to merchants because of the extremely low transaction fees (flat $0.25 per transaction vs Paypal's 2.9% + $0.30 per transaction).
However, as pg said, it is hard to encourage adoption when buyers only use it if merchants support it and vice versa.
Also, mobile payments show a great deal of promise.
I recall when I was young that sears started discover card, and it wasn't terribly long from the first time I heard of it to the first time I saw logos on windows and people accepting it.
At the time discover had no annual fees, and 'cash back' which I recall both being novel at the time (I was young though, so I don't recall too specifically). I think consumer demand helped push the vendor market, as well as Sears itself.