I've been contemplating this same question a lot of the past few months. As many readers point out, the type of application you're writing weighs heavily on what language and tools you should use.
The application I'm writing is a persistent world server for a game. The world server will need to be composed of multiple servers cooperating. Here are the languages I've been looking at:
- Erlang. I really wanted to use this, but just felt so constricted not even being able to do a loop. I realized later after much work that this feeling is simply because I hadn't (and still haven't, sufficiently) manned up and become comfortable with functional programming. The OTP framework is beyond compare for quickly building reliable distributed systems. Actors are a perfect fit for being part of the puzle for this target application of writing a distributed game world.
- Scala. Like erlang, plus the ability to use awesome java things such as Netty and Zookeeper. Lacks OTP, so a lot of the things Erlang has built in like distributed actors are missing. The Akka project seems to be targeted at bringing the missing things into Scala. For a while I felt that I had finally found the perfect platform for doing my server: Scala + Akka. But then I started finding that I was going to have to roll a lot of my own things (e.g. http://stackoverflow.com/questions/3692680/scala-akka-how-to...). Plus I started feeling like the Scala language is kind of more complex than it needs to be. Perhaps I wouldn't feel that way after working with it for a few months every single day.
- Clojure. The main appeal to me here is that I've long felt that I'd never be a real man until I could shoot lisp from my fingertips while laughing at all mere mortals. I've started studying lisp, including working my way through the MIT opencourseware. This is an ongoing side project. But Lately I've been thinking about just writing the software instead of trying to prove myself to the faceless elite whom I'll never meet. The most unique awesome power of Clojure to me for this application is STM, which I can get with Scala + Akka in a format that's less alien to me.
- Go. The syntax is easy. It's trimmed down and simple. Concurrency is trivial to do. There's a lot more rolling-your-own for this app, because it doesn't have built in OTP-like things for restarting failed processes, like Eralng and Scala+Akka do. What Go lacks in requiring me to roll my own, it makes up for by being lean and mean. Biggest downside to me about Go is that it seems relatively complicated to use external libraries I may want to use, such as Zookeeper. I also really grew to love Netty while using Scala, for handling tons of socket connections all at once. But perhaps I won't even need that when using Go, I'm not sure yet. I also think I'll be able to do a lot more per-megabyte of RAM using Go than I can with Scala. I have not tested that hypothesis, but plan to.
Various Python things. I considered using Stackless, but this seems like a dead end today compared to where it was a few years ago. I also looked at Twisted and worked through a book on it. I decided that if I want to do a lot of Arcane backwards tickery, I may as well use Erlang or Scala. Twisted, to me is the worst of all worlds: having to do strange arcane things, plus having it be in Python when it's done instead of compiled or JIT like Go or Scala. (Erlang gets a special pass in this context because it's so proven.)
Node.Js - I never really seriously considered this, because I really want to use an actor-style approach. (Maybe I missed the boat here, and should try this, spawning one node.js per cpu core per machine?)
Over the last year, I keep moving between Erlang, Scala, and Go, trying to figure out which I really want to use. My current plan is to write a working, multi-node proof of concept in all 3, and see which codebase I like best, then move forward.
Do you have any comments or thoughts I should consider along the way? Am I seriously missing out by so casually dismissing Node.JS for this application?
Plus Go has easy conurrency via message passing, which is nice to have these days.
Indeed. I voted Go because learning it made writing concurrent software FUN! (As it should be)
These 'wrinkles' are not fundamental problems with the language though, and most are on the slate for fixing, so it's just a matter of time for Go to be ready for anything. Development pace is still quite furious and improvements are coming out every week.
In the meantime, I thoroughly enjoy using it for smaller tools and assorted bits and bobs. It's a very pleasant language to work with.
But, really, good systems are based on well-defined protocols, which makes the language choice moot. (If you're passing Java or C++ objects over the wire, shame on you!)
I, for one, would really love to learn more about Go.
I'm interested in Haskell - what exactly is lacking about it? Is its design fundamentally flawed, or does it just lack a community and all the things that come with that (libraries, support websites, etc.)?
Even thought it is a mainly a functional programming language it allows the programmer to express in a more natural way lots of programming algorithms.
It allows a more easy way to explore multiprocessor programming by contrast with Go's go routines, because of the functional semantics.
If you're doing web development, or network applications, NodeJS is also notable, there's a reason it's getting a lot of attention lately, the fun-factor is just off the charts.
Haskell has lots of fundamental weaknesses when it comes to long-running real world programs, laziness being the most obvious one.
I'm sorry that I cannot remember the specifics of my problems, this project was done years ago.