Hacker News Reader

Comment by dandelany | Hacker News Reader

This is an excellent comment that clearly comes from a lot of experience. Can you say any more about why having multiple locks is such a red flag? Is it because you start getting into deadlock/hungry philosophers territory? Are there rules of thumb for using multiple locks that make it more tractable or is it just always a bad idea?

View on HN

Not OP, but that might be an implication. Another, more general reason is that this kind of concurrency requires design. It can’t be thought of as a defensive programming technique. Locks are too powerful and they are by definition a point of contention.

I think providing a lock is like giving up control to someone else. It’s inherent coupling. To contrast, when I first learned about them I thought of them as a protection mechanism of an isolated part, but they are an execution contract that everyone participates in.

"It’s inherent coupling."

This matches well with one of the beliefs I've been polishing up over the past few years which I don't think is well understood, which is that structured programming tends to create more coupling than we realize in deep call stacks. When you end up with a 2000-line call stack in Java, that's 2000 function invocations coupled together by the way structured programming works; for instance, at a bare minimum, that's 1999 functions that can't progress until the innermost one does, and that's not the only form of coupling.

I think the coupling induced by structured programming is fairly light per stack frame on average, but they tend to add up quickly because they're just so darned easy to add more of.

Locks are another thing that makes this really come to light. The more stack frames between some function and something above it in the stack that has taken a lock, the more likely it is for execution to eventually wander back into something trying to take a lock inadvisably.

I mean, I've had a number of deadlocks just within an object where it has some internal lock, and a method takes that lock, then tries to call a method that takes the same lock. Whoops. Easy to fix, of course, but that's the easy case. The cases get harder in real code.

So the stack is a 'lie'? Or the 'wrong' data structure? It isn't just caller knows about callee, there's cross frame and cross stack coupling going on. There is something interesting in your idea.

I don't think it's a lie. It's very, very useful. It did not survive as long as it did without providing value. It killed its competition stone dead for a reason.

I just think we underestimate the coupling it introduces. Doesn't mean the solution is to throw it all out. I'm not even sure there is "a solution". Just something to keep in mind.

The rule of thumb for multiple locks is always acquire them in the same order. Lock A, lock B, unlock B, unlock A. Otherwise you risk a deadlock. If another thread locks B, then tries to lock A, you're in trouble.

And keeping that discipline over many developers, over many years, is in practice impossible, which is why multiple locks seem attractive but are actually a major problem for complex systems.

You can, of course, encapsulate such a lock order with dedicated classes (or functions/closures) modeling linear types enforcing the sequence of lock A > lock B > unlock B > unlock A. The resulting objects can still be misapplied, but should fail immediately with a runtime error, making it much harder to inadvertently misuse.

But that still doesn't scale up to more locks of that size very well, because you start trying to take even more of them, and in dynamic orders, and etc etc.

I know about the theoretical solution of taking locks always in a defined order, but I don't consider it a practical solution. If taking locks willy-nilly lets you scale to program size X, this might give you 2X or 3X, but that's not nearly large enough in practice.

& dandelany, the community has chipped in and answered for me. I endorse this subthread, at least, as I write this comment. :)

IME there's plenty problems where nested locks will lead to a simpler solution than trying to avoid them. It's important to avoid when reasonably doable, but it's also not uncommon to take it too far and end up with a slower and way more complicated solution.

love when that happens :) Thanks!

The problem here is when you have more than 2, and especially as it combinatorially expands as you get more and more locks. Now I need Lock A, Lock C, and Lock F, but then this other critical section needs Lock A, Lock D, and Lock F, and if you are encapsulating then you start needing a whole lot of different functions for all the different combinations, and have to make sure that all are consistent.

I've got multi threaded C++ in production right now, and it is because the code is small and avoids multiple locks that I have any faith in it. As a code base expands, in kloc and in time, this gets harder and harder to reason about.

You can break this on a live system if you're in a debugger and manually lock one of the inner locks.

danluu claims there's other reasons it's not good enough but I was never able to figure out what he meant. Exception safety maybe.

Reply on news.ycombinator.com