The Erlang Challenge
yarivsblog.com
yarivsblog.com
I guess this just proves people read their biases into whatever they encounter. Like many, I thought the Arc challenge amounted to a test of how much a language was like Arc (guess what, Arc wins). So I assumed your post had to be satire.
And -- maybe the requirement of spawning a million threads doesn't seem like a cruel joke to you, but it is to anyone who has to work in a language other than Erlang.
A few other languages, including Scala and Stackless Python, have lightweight concurrency capabilities. AFAIK those languages do allow you to spawn millions of processes (or actors), but they don't handle garbage collection, scheduling, IO, distributed programming and native code interfacing as well as Erlang does.
If you discuss two different computer languages in any piece of writing on the internet, a large majority of the readers will assume you are advocating one over the other. I've seen this so many times already.
I'm just sad that I fell into the same trap.
Of course, the budget and the legacy code will have to be a very explicit part of that problem.
Meanwhile, this Erlang Challenge only tends to emphasize my impression -- which is just a stereotype, really -- that Erlang is certainly not the hundred-year language: it's a special-purpose language that is brilliant at a narrow class of problem. I'm convinced that, if I have some process that I want to spread across half a million processor cores, I should ask my company's Erlang guru to work on it (because, come on, if you can afford half a million processor cores you can afford at least one Erlang guru). What I'm not yet convinced of is that a world with N programmers who work in general-purpose languages needs more than (0.005)N Erlang gurus.
The challenge of the Erlang advocate is not to convince me, over and over, that Erlang wins its class; the challenge is to convince me that Erlang's class of problem is so important to my life that I should study Erlang rather than vascular surgery, or television repair, or other obscure technical skills that I don't know that much about.
I think it's important because it allows a program to easier to scale out rather than scale up. We'd want speedups in this way because CPUs are becoming multi-cores, and to take advantage of them one will have to write some type of parallel program, since it's been difficult to fully automate parallelizing serial programs.
In addition, parallelized systems (not just erlang) can be more fault tolerant and can fail more gracefully (or hobble along, if you'd like to call it that). Sensor networks are one example. Instead of a single radar to detect the environment, you many sensors out there, and they network themselves and report what they see/hear. If some of them gets destroyed that's ok, because the system's still functioning with less sensors.
So what concrete examples of problems fall into this class Erlang is good at? The obvious ones are the embarrassingly parallelized algorithms, like genetic algorithms, neural networks, 3d rendering, and if I'm not mistaken, FFTs. Indexing web pages is another. That said, there are problems that are inherently serial. It takes 9 months to begat a baby, no matter how many pregnant women you have.
Ruby and Haxe language writers are both implementing the actor model like Erlang, if that's any indication of how important they think it is. While I don't think Erlang will be the 100 year language, the ideas by which it's a poster child will reverberate in the descendant languages for a long time to come.
While erlang is a nice language, saying that it runs faster on multicores is just a lame excuse.
So it would be kinda like if I was a rolling a 10 meter (radius) snowball up myself and it takes me 1 hour, but if I split up the job amongst 9 other friends to get another 10m snowball, it would take each of us 1 hour instead of just 1/10 of a hour to roll up a tenth of that in volume (a 4.6m radius snowball)
But if it doesn't take longer to do 1/10th of the job if I split the job up 10 times, then erlang would be a good fit. There are problems that cater to that, and those that don't. Pick the problems that do for this technique. These are the embarrassingly parallel problems that I mentioned earlier.
So yes, just because you have multi-core, doesn't guarantee speedup. However, if the speed of a single CPU caps or slows in the future, and the number of cores increase on a chip, then us programmers have better have a good way to take advantage of that in our programs. Erlang and its actor model is one way.
The Erlang challenge highlights what Erlang is good at, but it's not meant to convince you that Erlang is good at everything. I've never claimed that this challenge proves you should use Erlang to solve any problem that comes your way.
The thing is, the Arc Challenges (I presume there will be others...) are explicitly intended to convince us that Arc is an awesome language for solving general computing problems. (Of course, they're also tools for designing Arc: If Arc proves unconvincingly awesome at any point, the plan is to twiddle its design knobs until the awesomeness returns. We will see how it goes.)
When you adopted the Arc Challenge's title and style, it left me wondering if you were aiming at the same goal. If you were, I wanted to invite you to shoot again, because you kinda missed. However, it seems that "convincing me that Erlang is good at everything" was never your goal at all, so I'm sorry to have interpreted it that way. (My interpretation may have been unduly influenced by the original title of the submission...)
By the way, although I didn't expect people to extrapolate from the Erlang challenge that Erlang is a good general purpose too, I do think that it is, at least for building backends for a wide range of applications. I wrote a followup on my blog to the Erlang challenge where I listed some of them.
There have been proposals for some way to mark user-defined functions as side-effect free and thus allow them to be used in guards. None has had acceptance from the core Erlang developers yet.
In any case this is not usually too much of a problem. The Erlang idiom is to use 'case' instead of 'if' in most cases (no pun intended) and where a calculated value is needed then just assign it to a variable beforehand - not quite as pretty I agree but no big deal.
case my_function(Foo) of
some_result -> do_something(A);
some_other_result -> do_something(B);
_ -> do_default_case()
end
Sony thought they could impose the PS3's multiprocessing regime on game developers through market force. But developers found the programming overhead unmanageable, which led to a less-than-compelling lineup, which failed to interest consumers, which means that now developers just work on the Xbox 360 version and port to PS3.
I was offered a job porting an existing game engine to the PS3 under the gun (the game had to ship in six months) and I would not touch it with a ten-foot pole.
Excuse me, at least in the imaging processing community we know very well what to do with all those extra cores. Particularly when processing 1 Gb images, and/or a couple hundred thousand 2048x2048 16-bit images.