>It is amusing to consider how much of the world you could serve something like Twitter to from a single beefy server if it really was just shuffling tweet sized buffers to network offload cards. Smart clients instead of web pages could make a very large difference. [1]
Very interesting to see the idea worked out in more detail.
[1] https://twitter.com/id_aa_carmack/status/1350672098029694998
Except that's not what it is doing at all.
It assembles all the Tweets internally, applies an ML model to produce a finalised response to the user.
It’s like me running a web crawler on my phone and saying I can replace Google.
The hard part is in being able to translate a search query into a list of pages.
And that requires a level of sophistication that far exceeds a laptop.
Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
It's akin to saying the magic behind OpenGPT is the dataset.
The comparable goal to the article is to be a search engine, not to fight google for best results.
George Hotz, is that you ?
Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw.
I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
I sometimes wonder how much value ML provides vs a proper sort function for anything but advertising.
Do you think that the most successful web companies in the world with arguably the best people i.e. Amazon, Facebook, Instagram, TikTok, LinkedIn, Pinterest, Youtube, Netflix, Snapchat etc. have no idea what they are doing. That the highly complex, expensive and latency impacting recommendation systems could be replaced by trivial sorting.
Or maybe they do work, do translate to increased usage and do significantly impact revenue.
Would it be better to live in a world where Twitter (for example) existed because it is a useful thing and not because it might make lots of money?
It's not just ads, it means the set of people you follow becomes extremely critical for your experience in a way that makes it far less engaging.
That may be good or bad for you as a user depending on what you want, but for Twitter having most people stick to the ML augmented timeline is essential to keep you hooked.
I strongly doubt that entire datacenters need to be used if and only if Twitter obsessively optimized for hardware usage efficiency over everything else. In reality they don't and make some pretty big compromises to actually get stuff built. Hardware is cheap, people are not.
b) No one has said Twitter operates entire data centres.
c) You need more than just NICs and ML accelerators to built a Twitter timeline. You need to rank the content, determine appropriate ads and combine them together. You can't do that in your network card.
They do though. 3 in the US alone (https://www.datacenterdynamics.com/en/news/report-elon-musk-...).
Twitter operates a handful of datacenters because their scale is such that it makes sense.