Look at early WhatsApp until they were aquired.
Vastly more innovative and scaled crazy fast without fail whales or anything.
Or look to Telegram today. Delivering a vastly more complex product with a fraction of the company size it seems.
Vastly more innovative and scaled crazy fast without fail whales or anything.
Or look to Telegram today. Delivering a vastly more complex product with a fraction of the company size it seems.
The difference between a messaging app and a microblogging app is basically the difference between an O(n) problem and an O(n^2) problem.
If you don’t think that adds some complexity, you’re kidding yourself.
But have you looked at the scale of what Telegram provides both in width and at scale?
Certainly there are celebrities with more followers on Twitter than the largest Telegram channels, but Telegram scales surprisingly far, and I haven't seen it struggle more than once or twice since the start.
It's clear you are totally clueless about how many features telegram has.
Scaling this is easily done with decomposition and sharding coupled with a suitable key->value mapping of external id to current shard. I first sharded e-mail delivery and storage for millions of users 23 years ago. It was neither hard nor novel to do then, with hardware slower than my current laptop handling hundreds of thousands of users each.
Do you believe that every Twitter user has an inbox stored on disk somewhere that just contains every tweet posted by someone they follow?
You also don't need to store every tweet, you need to store the id's of the tweets (a KV store of the tweet id to full tweet is also easy to shard), and since they're reasonably chronological the id's can be compressed fairly efficiently (quite a few leading digits of tweet id's are chronological).
You also have straightforward options for "hybrid" solutions, such as e.g. dealing with extreme outliers. Have someone followed by more than X% of total userbase? Cache the most recent N tweets from those accounts on that small set of timelinesyour frontends, and do joins over those few with users who follow them.
Most importantly, it's an extensively well tested pattern in a multitude of systems with follower/following graphs whenever consumers/reads dominate over a period of decades at this point, so behaviours and failure modes are well understood with straightforward, well tested solutions for most challenges you'll run into, which matters in the context of whether it'd be possible to build with a small team.
Put another way: I know from first hand experience you can scale this to millions of users per server on modern hardware, so the number of shards you'd need to be able to manage to deal with Twitter-level volume is lower than the number of servers I've had ops teams manage (you'd need more servers total, because your read load means you'd want extensive caching, as well as storage systems for e.g. images and the like - there's lots of other complexity, but scaling the core timeline functionality is not a complex problem)
Twitter is a 'universe of 100M connected people'.
WhatsApp mostly connected single entities together.
So, for example, 'real time search' and 'relevant updates'.
Imagine taking a firehose of 100M people's random thoughts, putting that into an index, making it instantly searchable. Now pull up the most relevant thoughts from those 100M to each and every other 100M user.
Now moderate all of it in really subtle ways, whereupon most of the 'negative activity' is tantamount to spam or annoying behaviour, and not anything we might normally consider 'abuse'.
That's an incredibly different challenge and that's only two small artifacts of what they are doing.
Twitter is not rocket science, but it's not trivial either.
Also consider that R&D is usually maybe on 20% of overhead - yes - it takes 'all those other jobs and expenses' to run a company.
Some of these statements are a bit glib.
Look at what they didn't do, which was make any revenue, or support any customers, or moderate any content.
I know I at least was shouting at them to take my money: it was the perfect HN product, reasonably priced, technically superior and with no ads or tracking.
And yes, I paid as well.
https://blog.twitter.com/engineering/en_us/topics/infrastruc...