Comment by byuu | Hacker News Reader

Comment by byuu | Hacker News Reader

Please don't be upset with me for asking ... but I am sincerely curious: I know that an MPMC lock-free queue is a vastly challenging thing (the best I could do lock-free myself was one producer thread, one consumer thread), but is this really the minimum level of complexity needed to implement this idea? There's a lot of comments (which is great), but even with that, the code just seems incredibly massive to me for what it is (over 2800 lines and 100KB, of course it has a lot of great commenting in there as well.)

If there really is no way to reduce the complexity (or at least without ruining the performance), then I understand. I know code size isn't a particularly important metric, I just find code that's been reduced to its raw essence a lot easier to understand and learn from than over-architected, over-abstracted code.

I ask because this is something I'm interested in understanding (I want to write a nice thread-safe allocator), but the code even with the comments and article is fairly overwhelming to me at the moment.

Either way, great job on making this! Really impressive performance metrics, too.

You're certainly right that it's not the minimum amount of complexity. But it's not that hard to implement a simple MPMC queue (e.g. if you can assume DCAS on your platform) -- what's hard is making it fast. So part of the complexity derives from a design that calls for blocks of elements instead of a simlpe linked list, for example; but a lot of the speed comes from that design too.

The rest of the code size (and complexity) arises from the feature set (which I intended to be fairly complete): Users can provide tokens for better performance -- but don't have to. There's bulk methods. There's a move constructor and assignment operator (for the tokens too!). There's traits so that you can override the malloc/free that's used, and all the constants. There's memory debugging machinery (hidden in an #ifndef NEBUG). It all adds up, but not without reason, I think :-)

I will add a "how to read the code" section to the README later on when I have a minute. I think it would be instructive for those curious.

The code has a huge amount of opportunities for re-factoring and cleanup. Some functions are massive.

Br, Robert C. Martin

It's not a proof of concept, it's an actual thing that's designed to be used. If you read the README, it's clear that there are a variety of nonessential features included which you may or may not want, but I see no evidence it's "over-architected."

Reply on news.ycombinator.com