Author of the take-home here: That's quite a good cycle count, substantially better than Claude's, you should email it to performance-recruiting@anthropic.com.
4,753 karma · joined January 22, 2012
The comments that I think aren't contributing much are ones that mention features that I didn't cover but make no attempt to argue that they're actually hard to implement efficiently, or that assert that because I didn't implement something it isn't feasible to make as fast as I calculate, without arguing what would actually stop me from implementing something that efficient. Or ones who repeat that this isn't practical, which I say at length in the post.
My joke fantasy startup is a cloud provider called one.computer where you just have a slider for the number of cores on your single instance, and it gives you a standard linux system that appears to have 10k cores. Most multithreaded software would absolutely trash the cache-coherency protocols and have poor performance, but it might be useful to easily turn embarrassingly parallel threaded map-reduces into multi-machine ones.
- spam detection: I agree this is a reasonably core feature and a good point. I think you could fit something here but you'd have to architect your entire spam detection approach around being able to fit, which is a pretty tricky constraint and probably would make it perform worse than a less constrained solution. Similar to ML timelines.
- ad relevance: Not a core feature if your costs are low enough. But see the ML estimates for how much throughput A100s have at dot producting ML embeddings.
- web previews: I'd do this by making it the client's responsibility. You'd lose trustworthiness though so users with hacked clients could make troll web previews, they can already do that for a site they control, but not a general site.
- blocks/mutes: Not a concern for the main timeline other than when using ML, when looking at replies will need to fetch blocks/mutes and filter. Whether this costs too much depends on how frequently people look at replies.
I'm fully aware that real Twitter has bajillions of features that I don't investigate, and you couldn't fit all of them on one machine. Many of them make up such a small fraction of load that you could still fit them. Others do indeed pose challenges, but ones similar to features I'd already discussed.
I'm now somewhat confident I could implement this if I tried, but it would take many years, the prototype and math is to check whether there's anything that would stop me if I tried and be a fun blog post about what systems are capable of.
I've worked on a team building a system to handle millions of messages per second per machine, and spending weeks doing math and building performance prototypes like this is exactly what we did before we built it for real.
I'm talking about a hypothetical HTTPS server that used optimized kernel-bypass networking. Here's a kernel-bypass HTTP server benchmarked doing 50k new connections per core second while re-using nginx code: https://github.com/F-Stack/f-stack. But I don't know of anyone who's done something similar with HTTPS support.
Hashtags are a search feature and basically need the same posting lists as for search, but if you only support hashtags the posting lists are smaller. I already have an estimate saying probably search wouldn't fit. But I think hashtag-only search might fit, mainly because my impression is people doing hashtag searches are a small fraction of traffic nowadays so the main cost is disk, not sure though.
I did run the post by 5 ex-Twitter engineers and none of them said any of my estimates were super wrong, mainly just brought up additional features and things I didn't discuss (which I edited into the post before publishing). Still possible that they just didn't divulge or didn't know some number they knew that I estimated very wrong.
I also bet that mainframes have software solutions to a lot of the multi-tenancy and fault tolerance challenges with running systems on one machine that I mention.
I actually ordered a plain steel plate first, but I realized that given that I needed them in a different position to travel compactly than the wide position I like for typing, I wanted them to snap in consistent positions so it wasn't finicky to line up exactly how I liked them.
I'm curious exactly why A* failed here. It worked great for me, as long as you design a good heuristic. I imagine it might have been complicated to design a good heuristic with an expanded move set. I see autochrome had to abandon A* and has an explanation of why, but that explanation shouldn't apply to difftastic I think.
It's unfortunate how https://github.com/janestreet/magic-trace and PMUs in general can't be used by lots of people using cloud VMs.
It's a CRDT in Rust and it uses a lot of similar ideas. Raph and I had a plan for how to make it fast and memory efficient in very similar ways to your implementation. I think the piece I got working during my internship hits most of the memory efficiency goals like using a Rope and segment list representation. However we put off some of the speed optimizations you've done, like using a range tree instead of a Vec of ranges. I think it also uses a different style of algorithm without any parents.
We never finished the optimizing and polished it up, so it's awesome that there's now an optimized text CRDT in Rust people can use!
The reddit comment I link contains an implementation that allegedly does arbitrary range queries, but it's nigh-incomprehensible so I can't tell how or why it uses 3 arrays.
It may in theory be possible to generalize the in-order layout I talk about in a similar way, but I'm not sure it would be that useful, maybe it would allow you to append rows or columns to your mipmapped image more easily, but I don't know of any applications where that's useful.
I'm biased on this though, as someone this might impact (https://news.ycombinator.com/from?site=thume.ca). From talking to other technical bloggers the consensus does seem to be that when we put a lot of effort into a technical article it nearly always makes it to the top of https://lobste.rs/ and /r/programming because it starts on the front page there but will sometimes flop off new on HN and maybe only make it months later if someone else resubmits the post.
There's definitely potential to do well on latency, if for example you have a server in NYC and your client has FIOS the inherent network latency could be 3ms and the only challenge is the encode/decode latency. It's possible Mighty has done something better, but every other remote desktop system I've tested spends more time on encode/decode than in the network, while claiming they're great (without giving numbers).
If you have figured out encode/decode latency, show me a high speed video (including the user's hands, not a screen recording) of say a Macbook on residential internet in NYC separate from your servers clicking things, compared to that Macbook running Chrome locally. Your numbers will almost certainly be worse for local interactions like typing in a text box, but you can show how it's better for things like clicking links.
Another issue other desktop streaming systems have is that video compression makes text ugly, especially when scrolling. This isn't as big of a deal for the game streaming systems but is noticeable on a retina display desktop. It's plausible Mighty has the codec settings cranked up enough so this isn't an issue though.
As far as I can tell the reason they don't raise prices is that the PR hit they'd take from all the gamers hating them for it would be a bigger deal than the additional revenue.
My experience with process-based parallelism is that yes on Linux it's basically isomorphic to thread-based parallelism. It's just so much more code to do the same thing.
In Rust adding a new special-purpose background thread with some standard-library channels is 30 lines of code and I can probably even access the same logging system from the other thread.
If I wanted to do that with processes I need to:
- Coordinate a shared memory file over command line arguments or make sure everything is fork-safe
- Find a library for shared-memory queues
- Deal with making sure that if either process crashes the other process goes down with it in a reasonable way.
- Make sure all my monitoring/logging is also hooked up to the other process.
If I want to use a shared memory data-structure with atomics I need to either not use pointers or live dangerously and try and memory-map it at the exact same offsets in each process and ensure I use a special allocator for things in the shared file.
Yes you can do all the same things with both approaches, I just find threads take way less code. It's not too bad if all your processes are doing the same thing, and you also need to scale to many servers anyhow. It's more annoying if you want to have a bunch of different types of special background processes.