As I read your design, one of the biggest problems is that to build up the infrastructure I think (and infer based of my experience with designing these kinds of systems) you are describing is that it's a massive engineering effort in and of itself. You basically end up rewriting a realtime-optimized BigTable or HBase, and that's much to much to expect of Twitter's small engineering team.
People seem to forget that scalability is a tradeoff, like any other aspect of a service. You devote an amount of resources to the problem that makes sense to your business. Despite twitter's downtimes, people don't seem to be leaving the service, despite several competitors. The service is down, but it isn't down so much that it's unusable.
For a site that's grown at such a rapid pace and has such a small engineering team, I think Twitter's scaling plan is probably very reasonable. They're growing their capacity at a rate they can afford, and a rate that they know will retain most of their userbase.
It is certainly reasonable to suggest that Twitter is good enough as it is because people still use it, though the grumbling in the market suggests that that may not be something they can bank on forever.
And have you checked latency serving out of EC2? Our experience is that the connections are fairly fast but the latency establishing the connection is significant, especially from the west coast where twitter is based.
Regarding EC2, latency isnt really an issue because Twitter is far from real-time. That said, haven't really noticed any problems with EC2 connection latency, but we are not live yet so I would defer to you on that.
If the average number of recipients grows along with the number of users, then there's a problem, because average number of messages should be growing linearly with userbase, so the total hardware required is quadratic. No way that would work out economically...
Basically you end up with this equation
cost = ((NumOfUsers * avgMsgsRevceivedPerUser)/maxWritesPerCPU) * costOfCPU
There is nothing quadratic here. Your are able to scale linearly with the number of messages received in the system. There is a fixed cost per message received which is exactly what you want. By the way, this is exactly how the internet mail system works and it is why it is so scalable.
E-mail's actually a really great example. It worked great as long as it was used for person-to-person communication, which held both the number of messages/user and the number of recipients/message constant. Then spammers found out that the more people who had e-mail, the more they could send to at once, and sent recipients/message through the roof. In 1997/98, there were actually a number of articles about how the e-mail architecture was in serious trouble, and unless some way could be found to limit spam, it would collapse within a couple years. Of course, they dealt with it through spam filters and blacklists on the server, so now e-mail works fine.
Twitter's problem is that both Messages/User and Recipients/Message average significantly higher than e-mail, and yet the resources they have available are significantly lower than the full resources of the Internet. So even if they don't have an architectural scalability problem, they could very well have an economics problem.
This doesn't even get into the complexity of scaling your model from 1,000 or 10,000 users where your brute-force methods seem like they will work and applying it to 10,000,000 users where your model will die a horrible death, kinda like Twitter is today.
That doesn't make your opinion of what's wrong with Twitter any more valid than the crap TC keeps printing unless you happen to also have lots of internal details of the Twitter architecture.