Hacker News Reader

Comment by watson | Hacker News Reader

Moore's law doesn't concern it self with clock speed, but with the number of transistors. While the clock speed have stayed about the same, the number of cores have increased. The problem is that it's really hard to make a program that takes advantage of all those cores, so you as a regular consumer doesn't see any difference in your day to day usage.

View on HN

One thing I find myself asking a lot is if it's really hard to make a program take advantage of the cores or if the prevalent tooling and paradigms of today make it hard to take advantage of the cores.

While most programs are not embarassingly parallel in nature, I suspect the latter plays a bigger role than most of us tend to admit.

I think you are exactly right. I've done a lot with concurrency, and the fact is that there is no life vest so you better know how to swim. I think an architecture made specifically for programs to take advantage of concurrency would help an enormous amount.

Right now we are two steps removed. There aren't even many good data structure implementations out there. There is moodycamel::concurrent_queue and different memory allocators like jemalloc and the new rpmalloc, but the vast majority of programmers likely don't even realize that malloc locks and kills concurrency. Maybe they grab something from boost that just surrounds a data structure with a mutex, which is lazy bullshit.

I actually think the majority of software that runs slow can be made to be sped up nearly linearly using dozens of threads, but the few niche libraries that help are overly complicated to compile and use, bloated with enormous dependencies and end up being a minefield of usage. At the moment many people's idea of multi-threaded is mostly fork-join techniques like openMP and that will never scale to using all cores throughout the whole program, since is one very narrow technique.

As someone with only a basic understanding of threads, what is a better idea than fork/joins for scaling? Are you talking about thread pools or something else?

Yes for thread pools instead of forking (could be process pools too). But that's only scratching the surface. It's how you communicate with those threads and allocate their work that makes the difference.

I think the quick and dirty answer is yes, although I doubt I would structure things the same way as a normal thread pool.

If you think about the heart of the problem, it is really synchronization, and that only needs to happen in certain places in programs. Anything you can break up into pieces that can be transformed independently can be leveraged for concurrency, which also implies that independent stages in a pipeline can be concurrent as well.

The particular implementation I had in mind is Erlang. Due to the green thread implementation, immutable data, and message passing, Erlang is able to offer a simple, robust concurrency platform. Of course it's still possible to write very sequential code but generally someone writing idiomatic Erlang won't have to go out of their way to get concurrency.

It is also far from ideal when it comes to performance from what I've seen. Maybe its overall design is good, but I don't believe that adding so much overhead over native programs is necessary.

From my somewhat limited reading it seems that the only inherent performance hit in Erlang's implementation is that all its processes can be interrupted externally, I think the term is cooperative multitasking?

Otherwise I think the rest of Erlang's performance is because it's optimized for reliability and distribution rather than raw performance.

I really think there's only two rays of hope for parallel programming.

One is promises (async/await). This takes the burden off the programmer for explicitly managing concurrency, and potentially allows lazy evaluation in many cases (if non-side-effecting). I've never worked in Lisps but it seems like a viable model in many ways.

The one I wanted to bring up was probabilistic programming. Imagine a tree where there's not just one place to put something, there's many, possibly distributed. As such you greatly reduce the chances of contention from multiple threads. They would also probably (almost necessarily) reduce the amount of rebalancing work.

GPUs seem like the perfect model here in multiple ways, not only do you have the massive degree of threading that surfaces any problems in such things but you also have the degree of threading to deterministically search these kind of structures. GPUs also highly discourage things like CAS or mutexing that we need to move away from.

(PS nice username)

I'm not sure how the tree you are describing helps anything. Reading can always be done concurrently, it isn't the problem - all threads can read from the same memory at the same time. The problem is writes, which means any alterations.

GPUs are useful for the kinds of parallelism that has always been known to be easy - fork join with all threads writing to new memory. There is no mystery to how to do this.

> CAS or mutexing that we need to move away from

compare and swap is the backbone of lock free programming. Mutexes to me seem to rarely be ideal, but there is no getting away from compare and swap as far as I know.

Reply on news.ycombinator.com