If concurrency is easy, you're either absurdly smart or you're kidding yourself
plsadventures.blogspot.com
plsadventures.blogspot.com
Concurrency isn't hard; Sharing is hard. This is the reason that mutable state hurts understandability - things can change under you because the view of state is shared. This is the reason global variables are bad - they are shared between different parts of the program, and can be modified from places you can't see. And this is the reason shared-memory concurrency is hard. Add to shared-memory concurrency a dash of race conditions in operations involving a load and a store, and things get hard.
Of course, I'm oversimplifying as I write this, but I think that's really the gist of it.
You get the same benefits as concurrency (maximum processor usage, at least when you only have one processor), but you avoid the pitfalls of shared memory, race conditions, and unexpected state changes (because only one event handler runs at a time). It does take a bit of getting used to (it seems so easy to make function calls like time() or mkdir() that are actually blocking and thus technically should be separated into a new callback function). But I find that the benefits of processor efficiency plus the comfort of avoiding concurrency problems are worthwhile.
A great talk about the advantages of concurrent programming in producing clear and understandable program structure can be seen in Rob Pike's newsqueak tech talk (http://video.google.com/videoplay?docid=810232012617965344)
I also think that making a distinction between concurrency and sharing is rather artificial. Embarrassingly parallel problems aside (where the concurrency is really better characterised as co-habitation of a computing resource), it's still pretty easy to get both shared-memory and channel-based message-passing wrong in concurrent situations. I agree that the presence of shared-memory makes it worse, and while my experiences using things like CML without using mutable storage have been much more positive than say... Java concurrency primitives, I think it's still "hard" in that it requires considerably more intellectual effort than writing non-concurrent programs of similar scope and size.
Linear typing can also help by enforcing ownership transfers between workers operating on data, allowing efficient data transfer without sharing.
or the way they are using concurrency is restricted to nice simple patterns of interaction to which we can apply some simple mental analogy
Actually this possibility covers most programming requirements. I like to think I'm superhumanly smart, and I would love to tackle a hard concurrency problem, but so far I have only needed mundane engineering ingenuity to map my problems to known solutions.
When people say "concurrency is easy", they mean "I use little bits of concurrency that I understand well and I find those easy". That's not the same as "concurrency is easy". That kind of arrogance is essentially like saying "flying a jet fighter is easy, because I know how to use a seatbelt".
However, the reality is that developers can not do this checking with complete accuracy. The simplistic reason is that the complexity of concurrent programs is exponentially greater than that of non-concurrent programs. In a concurrent model if I have m threads of execution and n read/writes per thread I need to verify n^m states in my code. Such a blowup in state makes any reasonable person look for generalizations and optimizations, but those then lead incorrect assumptions and ultimately defects.
Outside of this simplistic reasoning lie other problems which serve to exacerbate code verification during design and construction; realities of deadlines, access to source or documentation of components, the learning curve of technologies used as black boxes.
For example, is the process that results from parallel composition of two deterministic processes deterministic or non-deterministic?
The answer is "it depends".
Concurrent "deterministic" processes are not significantly easier to reason about (informally) than non-deterministic ones, whichever definition you choose.
In fact, re-reading, I definitely don't see anything in there that could described by either "determinism" or "non-determinism", so either your definition differs wildly from mine, or you're choosing to be dismissive deliberately :)
From the article:
Now, before you jump up and down and tell me that "insert-your-favourite-concurrency-model-here addresses these problems", I know there are strategies for managing the complexity of these tasks. I did concurrency theory for a living, so I know some approaches are better than others. My point here was essentially to illustrate why it is important to manage complexity when doing concurrency. It's definitely not just a case of "you can't do concurrency so that means you are stupid". I'm arguing that pretty much everyone is bad at concurrency beyond some simple patterns of interaction that we cling to because we can maintain a good mental model by using them.
To elaborate, things like the technologies you mentioned may provide better mental models for concurrent development, but I'm not convinced that they have solved concurrency in its entirety.
And I'm not saying they solve "concurrency in it's entirety". I'm not even clear what you're trying to express with that statement.
My only point is that the domains they are designed for are constrained. I don't mean this as a negative - I consider a general pthreads-style implementation of concurrency to be constrained by the ability of people to reason about that kind of concurrency, probably to a much larger degree than a considerably more elegant implementation in a modern functional programming language.
I would argue, however, that such languages simply make it easier to construct concurrent programs that follow certain patterns. They provide useful abstractions for managing the complexity of concurrent interactions, which I think was my original point, and which I think is brilliant! What is missing is something that provides good-quality abstractions, high-quality static analysis and very strong safety guarantees in the general case.
Think about things like mobile processes, "ubiquitous computing" and software with huge failure tolerances by virtue of being able to reconfigure and coalesce resources at will.
And hopefully the flow-on effect of that would be to essentially stop horrible concurrency implementations being used in languages like Java and friends.
The problem domain of concurrency?
"What is missing is something that provides good-quality abstractions, high-quality static analysis and very strong safety guarantees in the general case."
They have excellent abstractions and great safety guarantees. Not sure what you mean by static analysis. A concurrent program is non-deterministic, what is going to go under static analysis? I'm sorry if I'm misunderstanding something here.
"software with huge failure tolerances by virtue of being able to reconfigure and coalesce resources at will."
Wasn't this the whole point of Erlang? You know like telecom switching?
Concurrency is hard even when you're just using a pencil and a piece of paper. If you write down a description of a system using any kind of simple notation, it will be difficult for me or anyone else to read that an have a good intuition about the runtime behaviour of that system. This isn't a discussion of statefulness or mutability. It's just about the presence of a very large number of interactions that essentially defy any usual sense of causality.