Hacker News Reader

Comment by samsquire | Hacker News Reader

I've learned that You cannot accelerate writes to a single memory location by adding cores. Memory bandwidth is fixed when running on a single core, when you add writes from other threads you want thread safety, which slows things down. Mutexes do not scale. If you want to saturate the memory bus you can read and write to different locations and fan out.

I am working on a multithreaded barrier which does mass synchronization on a schedule, a rhythm.

It can send ~169 million messages a second across 10 barrier threads and 63 million event ingests from 3 external threads.

Your algorithm has to be redesigned to support this style of programming. Even message passing can be slow due to context switches. But bulk buffer processing is fast.

View on HN

The trick with scalable threading is to make the behaviour as independent as is possible. Including memory allocations. “jemalloc” been something worth looking at.

It’s better to follow the scatter/gather model of MPI (don’t use that) in the same process and do as much amortisation in each thread as possible before collation.

When it comes to I/O (with spinning rust), a measured rule is to use twice the cores to account for seek time and use big writes/reads rather than small ones. This can saturate the disks so do measure it.

I spent about ten years worrying about these kind of issues on enterprise storage in C++.

Would snmalloc (https://news.ycombinator.com/item?id=37851210) help with scalable threading or not? It claims to be better at allocating memory on a producer and freeing on a consumer thread, and "Freeing memory in a different thread to initially allocated it, does not take any locks and instead uses a novel message passing scheme to return the memory to the original allocator, where it is recycled. This enables 1000s of remote deallocations to be performed with only a single atomic operation enabling great scaling with core count."

It’s a shame that they don’t compare against mimalloc which is another Microsoft project and it’s unclear which tcmalloc they are comparing against (the gperftools one is stale and performs worse than the current standalone release)

I’ll offer that if you have a long lasting thread, you might consider allocating big blocks to it and having a “pool” it can grab memory from. In C++ you can use a shared pointer to release it back to the pool.

This avoids contention in user space. It also reduces fragmentation. You can also bound the memory usage by blocking until memory is free.

If memory serves, boost C++ has some code to help there though I did it myself.

Reply on news.ycombinator.com