Deterministic multithreading is hard (2024)
factorio.com
factorio.com
after HN, it's one of my favourite places on the internet because i constantly learn new, random, insane things almost every time. imho it teaches you how to think + shows you what great engineering taste looks like. sorry if i'm overly effusive but each post is so deeply technical and well-written that i can't believe it's free.
you don't need to know anything about factorio or gamedev btw (i don't), just pick a random number between 1 to 438 and start reading :)
Concurrency is hard. Blizzard added progressively more and more concurrency over time to rescue orphaned resources assigned to a single shard that was undersubscribed while another shard in the same AZ was seeing flash mobs. But the way they documented it was more of a tea leaves situation. Only enough data to guess what they had done if you were familiar with the space.
> DTHREADS works by exploding multithreaded applications into multiple processes, with private, copy-on-write mappings to shared memory... Experimental results show that DTHREADS substantially outperforms a state-of-the-art deterministic runtime system, and for a majority of the benchmarks evaluated here, matches and occasionally exceeds the performance of pthreads.
It’s a shame. The real world of development would be significantly richer if these ideas had better funding and dedicated long term development.
One of the researchers behind either rump kernels or unikernels talked to us here about making it usable. He said he was discouraged by his advisors from doing that. He could write more papers instead.
The reason they think that was is it's quantity over quality in much of academia, esp citation scores and funding for new research. Some groups seem to have their researchers use some of their time to build useful software. Most isn't production quality or maintained because they're basically paid not to do that. It's why many don't join academic research and others supported funding cuts or reform.