Threads, slow and buggy
and.org
and.org
Threads should always be an absolute last resort. Programmers working on concurrency, locking, etc are often solving self inflicted problems. I don't know why they do it.
edit:
I've mentioned it in the past, and I do hate harping on about mibbit ;) but I had a similar experience there - as you can imagine, a large amount of dns lookups (both forward and reverse) are required. First version used a single separate thread to do them. Had to do lots of synchronization in the callback. 2nd ver had a threadpool, which was even more complex. Still slow as hell.
Final version, and the version in use now uses async nio in the main network thread, and is as fast as fast can be. The code is far simpler/faster than the threaded version, and handles thousands of lookups with minimum of fuss. There's just an initial investment of writing dns packet+udp code (Not rocket science).
At first glance, threads often seem like the 'easy' option. Often they're not. In my example, it was far easier just to knuckle down and write a simple async dns client than constantly mess around with concurrency issues and try and get it to go fast.
The code in this article looks horrible to work with, and reminds me of how my first version looked :/
The simple solution is to requires any sharing of state to be made explicit (e.g., message passing). This obviously introduces some overhead, but I think it could eventually become like assembler v. higher level languages. A handful of very talented hackers will still be always be able to beat any threading abstraction by manually dealing with everything - but the abstractions will get more and more optimizations (e.g., copy-on-write variables, more static analysis) until you rarely bother looking underneath them.
As a trivial example (an example I included in the post above yours), consider 'copy-on-write' optimizations. Thread A can pass a message containing Object O (which is 50MB) to Thread B. Naively, this would be pretty expensive - however the compiler can set things up so that a copy of O isn't made until either B or A tries to write to O (in which case in can even try to just copy the part that was mutated). This provides a healthy balance between the raw performance of fork-style threads and the safety of multiple processes. This sort of optimization wouldn't be possible just using processes since, obviously, Process B couldn't (or at least shouldn't ;-)) directly look into Process A's memory space.
I never said don't share state - I just said don't transparently share state. Make it explicit and/or make it safe. As I said, it's never going to be quite as fast as raw fork-style threading but, with a sufficiently smart compiler, it can come close - and be a lot easier.
That is the reason languages like Limbo ( http://limbo.cat-v.org ) and Erlang were invented (and CSP, obviously).
They're moot points IMHO. Trying to make languages that deal well with threads/concurrency is a pointless exercise. What's the advantage to doing this, over just not using threads?
Message-passing works quite well, but coroutines are also a viable option. The problems come from using pre-emptive threads AND shared state, because reasoning about who is modifying what (and at what stage of completion) becomes difficult. Co-routines avoid the former, message passing the latter. Which is simpler overall depends on the problem.
Why not both as in CSP/Limbo? ;)
Edit: Note that you can get both in C too using the confusingly named libthread(2) http://man.cat-v.org/plan_9/2/thread (Also available in Unix as part of Plan 9 from User Space http://plan9.us )
Coroutines are in the core language, and there's also ConcurrentLua (http://concurrentlua.luaforge.net/) for message passing.
Not really, it is not too hard to implement it locally as passing pointers around.
Sadly this is true of most 'problems' in the software industry :(
I think your last resort suggestion is very much based on the notion that most of what we ever do is offloading all data access onto a SQL database. I like SQL databases, however unfashionable they may be at the moment, but they're not the solution to everything.
As memory gets cheaper it becomes more feasible to just keep everything in server memory and access it from multiple clients. You can't do that without dealing with concurrency issues.
I do have to admit that I didn't bother to dig through the final example looking for the errors, so perhaps the author did have a better argument in there but didn't bother to pursue it more aggressively.