>I don't see the parent suggest that you use treat coroutines like threads at all. In fact, he's explicitly suggesting that the same issues that exist with threads don't happen, or are at least significantly diminished.
Right, but by implication if you don't want them to happen and are afraid someone will override __iadd__, then by no means just use a lock. Both eventlet and gevent I believe will monkeypatch threading module to now become "green" threading module.
> However, since nothing appears to break if you get it wrong
But nothing would appear to break in the deferreds case I suggested either. It would still be accessing shared data in an inconsistent way.
> I suggest that there's way too much room for people to get it wrong there.
That is why a private queue with a green thread attached to it (a actor!) is a better way to build large concurrent systems.
> Furthermore, in many cases there are simple ways you can make a synchronous library look async in twisted: deferToThread is the most common/obvious one.
Yes it is there and it works but it is clunky. After switching to gevent we, for example, got about about a third to a half as much code as before. A lot it was boiler plate code. inlineCallbacks and yields. Our own counter-parts to already existing libraries to make a Twisted alternative.
> Can you give a practical example of where you used a semaphore to synchronize shared mutable state access? I've only used it for, say, concurrency limiting, I think
Pretty much the exact problem I described above. A server was processing requests from the user. It was listening on a TCP socket and reading commands delimited by a new line. One of the commands initiated a business operation consisting of multiple steps. That operation involved going to some databases, calling a number of sub-processes all of which was handled via callbacks. As the operation was taking place, a local piece state data was updated, tracking the progress etc.
Now if before that request finishes, a new command comes. And starts firing the same sequence of callbacks, but the previous one hasn't finished yet.
So that's it. You are accessing a shared piece of data which is now in a logically inconsistent state in respect to your internal business logic and you have no idea it happened (just like with threads).
So you need to acquire a deferred lock and then release it when done.
> I'm actually very interested in actor-based concurrency, and I think many of my fellow Twisted developers with me (we have a good contingent of actor + obj-capa fanboys). I have a few systems where the reactor is pretty much that queue for all intents and purposes :)
Yeah we switched to that. Actors are basically an object attached to a green thread that a main method. Actor has a queue. Outside processes can send it messages by putting them in the queue.. Actor in main method can do self.receive() for example to get messages off the queue. Basically what Erlang does. You have choice if you want to duplicate data (make copies before putting in queues) or use objects with locks to serialize access to them (say if they are large and copies won't work).