What is Twitter? I see it as two things:
1. Infrastructure: it is a means of sending thousands of short messages a second to people in asymmetric relationships. Technically this isn't a hard problem. There are many companies around who handle traffic many orders of magnitude above this. The danger with infrastructure is that it will get commoditized. The cost in terms of resources (bandwidth, CPU, power) is not that high;
There are parallels between Twitter and SMS. SMS survives and is hugely profitable because of the telco cartel that monopolizes it. Where hosting companies will provide you with bandwidth for cents per GB, SMS costs thousands if not millions of dollars per GB. That too will come to an end eventually. What saves it now is all the players want to push their own standards and they aren't interoperable. It's IM fiefdoms all over again.
2. An audience. Twitter and its users have the ability to reach a large number of people. Originally Twitter was envisioned as a means of real-time status updates between normal people. My impression is that this use case is basically dead. The vast majority of Twitter usage seems to be as a communication tool by celebrities. There is a business in that but it isn't the revolutionary communication tool it was once thought (and some still think) it to be.
So the camps in question (the API camp and the ad camp) fit into the above.
Owning infrastructure can be hugely lucrative (eg SMS). History is full of examples of this. Railroads, oil, telephony, etc. The problem is that you get too big and the government will intervene.
Messaging infrastructure is too important for one party to own it. Email is federated and open. The future of messaging is too (I believe) federated and open.
So that leaves (2), the ad camp. The problem with advertising on Twitter is the same as it is on Facebook: it's an unwelcome intrusion.
Perhaps Twitter could have (and may still be able to) build a display ad business but Twitter is still not a particularly mainstream product (and IMHO it doesn't look like it ever will be).
Perhaps Twitter could've had a subreddit/Facebook pages type hubrid model. Upvoting of Tweets is possibly interesting.
Of course this faces the same problem Digg had: few things are globally interesting. What's interesting to you is not necessarily interesting to me so you changing what I see is again an intrusion.
Basically "content gateways" (like Digg, Slashdot, etc) were I believe transitional. I don't believe you'll have a repeat of Digg moderators being paid to promote stories. You can (and people do) pay celebrities to retweet things but that still only goes to their followers. They have to consider their brands and building and keeping their respective audiences too.
Those thinking Twitter should have an open firehose (or close to it) are neglecting the reality that Twitter is a business and needs to make money somehow. Open access to their content greatly diminishes that.
Lastly, quite a few people will speak of "betrayed" by Twitter over API access. They also believe that developers were largely responsible for making Twitter popular. As I said (just yesterday) that this is inevitable. Become too successful and the platform subsumes you into their core offering (no one wants core features controlled by third parties). Developers were I believe largely incidental to Twitter's success. Twitter succeeded (as it were) because people used it. To take credit for that as a developer community is like saying that the automobile succeeded because of car wash stations and autoshops selling rims.
EDIT: one last point. It's popular, particularly on HN, to simply build an audience or something that scales without concern for monetization. I certainly understand the logic behind this. Sometimes it works but sometimes it doesn't.
Twitter hasn't come up with a scalable business model yet because there isn't one. As much as founders can be praised for "swinging for the fences" and going all in, sometimes the best outcome is to be acquired because what you've developed isn't a viable independent business but could be an incredibly valuable part of a wider portfolio of products.