I think this is probably right, but I'm wondering what else you've built in Erlang. Was it a good fit? Why or why not?
Katamari also has some fairly good fault tolerance characteristics. It aggressively tries to maintain a cloud and it has no real single points of failure. It recognizes hung or crashed instances of individual workers and restarts them. It also supports arbitrary versioned resources existing in parallel, which is great when you want to do A/B testing.
It was also designed so that it was usable in both a development and production environment. While currently little slow for most production uses (~2ms overhead for most queries, with a special case of ~12ms), it's plenty fast and runs in less than 10mb of memory on most linux systems, and can transparently go from a 1-box to a N-box solution.
Microsoft open-sourced it for me (after 2 years of bureaucratic nonsense), so I am just pulling out the useless and crufty parts before I put it up on GitHub. I've given 2 talks about the technology, one for Fuzed at Railsconf and one on Katamari at the last Erlang Factory. I hope to have a modernized version maintained this year, and start moving the tech forward.
Katamari is about 10k lines of code; with about 2/3 of that being Erlang and the remaining 1/3 being Ruby code for the glue (Ruby was Powerset's primary glue and binding language, so it is the only supported binding). Katamari aggressively leverages the characteristics of Erlang to accomplish this feature set; its safe to say all the fault tolerance wouldn't have been doable without Erlang's implementation providing the basics for it.
Chicago Boss, an MVC web framework in progress: http://github.com/evanmiller/ChicagoBoss
This doesn't get mentioned enough, but Erlang has excellent compiler tools, so if you're willing to roll up your sleeves, you can usually hack the language to fit your problem. For instance, Chicago Boss ships with a nice but completely non-standard database querying syntax, described here:
http://www.chicagoboss.org/api-db.html
You will weep for joy when you rewrite a gnarly SQL query with a few well-chosen Unicode set operators.
The compiler tools also make it easy to generate Erlang AST and compile it to BEAM (Erlang bytecode). That's how ErlyDTL works. I haven't done performance benchmarks myself, but the folks at Zotonic report that their Django templates and controller code come out 10X faster than comparable PHP code.
I think a couple of things prevent Erlang from being a do-all language:
1. Lack of libraries. The situation has been steadily improving, but Erlang's VM and processing model makes it difficult to draw upon the "C tradition" in the same way that Perl, Ruby, and the others can take a popular C library and write a thin wrapper around it.
2. Inferior OS integration, at least with UNIX. For instance, custom signal-handling facilities are non-existent, and you have to jump through strange hoops to do simple stuff like close only the write end of a read-write pipe, or get the exit code of an exec'd process. Weak sauce for anyone who has spent a lot of time with Perl, Ruby, or C.
Twoorl, an open source Twitter clone: http://github.com/yariv/twoorl/.
Vimagi, a graffiti-like facebook app: http://apps.facebook.com/vimagi-paint.
Also, parts of Facebook chat (I work at Facebook).
It was a tedious, frustrating experience, even though you'd think it would be the perfect language.
It ended up being too slow (probably because of all the string processing)
In my entire programming career, I've never before had to join a mailing list and ask questions to get things done, that should tell you something.
I think the first time I hit the mailing list was because I couldn't figure out how erlang can boot up and change its effective user id. (setuid). The reason being it needs to grab a privleged port as root. Well forget that, it has no idea what that means. The recommended way is to use iptables port forwarding. I think thats how couchdb does it (or one of those nosql things)
Logging also didn't seem to fit well with linux. You can manage to write some stuff to syslog, but erlang really prefers its confusing way of logging. See things like this http://www.erlang.org/doc/man/sasl_app.html.
Oh and if you decide to just write data to the error log in the simpliest way possible, like this error_logger:error_msg("An error occurred in ~p~n", [a_module]). Guess what, it doesn't open that log file in append mode, so it wipes it out every time you restart the erlang process. You have to write and install your own error log handler to open the file in append mode.
Want to do some dns lookups? Sounds simple. It starts up a gen_server thread... and starts caching the results! Talk about principle of most surprise. And it doesn't handle mx records. But if you get on the mailing list the guys will tell you there's an undocumented library deep in the distro that will handle this, and probably won't change. So you give up in frustration and just write a port to run dig.
It just went on and on.
You can really tell that erlang evolved to run self sufficient on special hardware and not in a supportive unix environment.
And of course there are the basic language frustrations. What bothered me the most was simply the lack of a 'return'. This meant you couldn't 'short circuit' a function, like "if something then return", and it often resulted in another level of nesting, even for the simpliest stuff. Yes I know that sometimes you could use function pattern matching to factor this out, but not as much as you hope. Having a 'return' statement in erlang is part of a religious war, which I'm not going to fight, but I'm strongly on the having side.
I found some erlang things worked great for my project, like the server and finite state machine 'behaviors', but when I realized the string processing was so slow I just switched to stackless python, wrote it in a fraction of the time, never used it, and ended up keeping the existing perl version :)
You can write a MTA in any language, so its probably not the best fit for erlang. The best fit for erlang would be something that only erlang could do, and just suffer through it ;-)
Well yeah, it was written to run telephony equipment. It's not a general purpose language, though it has been moving that way.
Now I've checked again, so let me summarize:
It started to spread "outside the lab" in 1987. In 1995 there were three commercial applications: network simulator, mobility server and distributed resource controller. First is a simulator, not running on the specialized hardware, the third was running on "a network of Windows NT machines." The second: "acts as a call control system when linked to the PBX"
OK, the second was running on the additional SPARC-based machine, not big iron, but AFAIK certainly more powerful than the PC's of that time.
So I don't see anything that was targeting really "specialized hardware" particularly not what we'd think as embedded (less power consumption, slower processors than the PC CPU's). It was a research project, and it had traditionally performance problems.
Even today, after so many years, when you compare Erlang HiPe (Hi Perf, you can imagine others then!) with something higher than C, let's then say Lua JIT it's easy to see which one is faster:
http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
Edit: Or for multi core 64 bits with Mono:
http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...
Erlang excels at distribution. People tend to forget its raison d'être is fault-tolerance (not concurrency), so when you set off working on an Erlang application you presume that it will run on 2+ machines. That's why it's great for things like Facebook Chat and Riak. In addition to those, it's also good for writing control and supervision layers, where workers are written in Python, Java, C etc.
While I'm at it, I'd like to plug http://veldstra.org/whyerlang/ It's in a real need of an update, and I'd love to hear any feedback.
Francesco
- An IRC bot -- great fit for the language
- A URL shortener service (three times) -- first on Yaws, then on Nitrogen, then on ChicagoBoss (Hi EvanMiller!)
Erlang is the most fun I've ever had writing code. While a lot of that is the functional language aspect, I find it an extremely intuitive language at least at a low level (once you get past the 'there are 4 ways to end a line?!?!' stage). I'm still getting used to thinking through bigger problems in an Erlang way, but thats more a function of my brain than the language.
I think Erlang gets a lot of unfair knocks given it's pigeonholed reputation for 'concurrency and fault tolerance'. Certainly it has it's limitations (libraries), but generally, I think its a pretty well-rounded general purpose language.
A game data aggregation platform and feed system for EA.
A lot of open source Erlang stuff (http://github.com/ngerakines)
It was a great fit. It lacks Unicode processing capabilities by default, but there are a few open source libraries for that. The only thing I dislike about Erlang is the poor string manipulation and lack of clean structures (Erlang's records are an ugly hack IMO).
Erlang is a great fit - works well, allows you to build a native stack without any cross-language impedance. Your application is cluster ready from the get-go and allows you to focus on the functionality rather than any of the utilities required to make the damn thing stay up.
Great fit. OTP process restart characteristics. Per-tenant process separation. Process abstraction, in general, rather than threads. Finite state machine behaviors. Live code upgrades. Mnesia database.
Only real downside was it took the rest of the team a bit of time to become effective in Erlang... about 4 to 6 months.
"Wings 3D is a subdivision modeller with an user interface that is easy to use for both beginners and advanced users"
http://blog.ezyang.com/2010/08/tour-of-preach-distributed-er...
Great stuff. Erlang may make you smile (again).
It's a reverse engineering of a popular MMORPG called Ragnarok Online (or RO). Last time I was working on it, you could log in with the official client, walk around, teleport, chat, etc. (The walking had to be done in C, though; one pathfind per walk request is not something I'd want to leave to Erlang.)
Vector Map Server - a BSc thesis project. Vector geospatial data streaming server, very similar thing to server's streaming vector data to Google Earth or Map24. Also works as WFS server. Has to handle a lot of data, but since data is mostly immutable it works well.
Backend service for not yet released liveblogging site. Done in collaboration with few other developers. Massive pubsub system. Done tests with 1M concurrent users on 200 small-node EC2 cluster with end-to-end latency mostly < 1.5s.
Texspace - spatial search server based on Microsoft Research paper. Fuzzy spatial search capable of working with completely unstructured queries. Had pretty big issues with processing, indexing and analyzing big amounts of data due to slowness of all DB solutions available at that time. Writing geometry algorithms in Erlang is not trivial due to immutability, since all of them are described in imperative languages.
IM subsystem for microblogging site (which is pretty big in Asia) - implemented 5 proprietary protocols. Erlang is perfect for such tasks! The only issue was lack of documentation for proprietary protocols ;)
Map routing server - implemented optimized A* search with route graph generalization. Writing fast A* is tricky in immutable world. Books like Okasaki's "Pure Functional Data Structures" FTW!
WMS Tiling / TMS service with caching. Simple and easy task for Erlang. Mnesia + file-level cache.
Overseer - Nitrogen + CouchDB app for tracking employees time by doing screenshots via VNC and marking them as "work" / "not work". A breeze :)
Erlang FTW!
It's a perfect fit.
Built github.com/RJ/playdar-core in erlang, after initially writing it in C++/boost
A options feed aggregation and charting backend.
A package management system.
many many other small apps.
Cheers, Martin Logan erlangcamp.com
They are also working to release a game engine (NAOS[2]) built from their server code, which also uses Erlang pretty heavily
[1]: http://www.vendetta-online.com/ [2]: http://www.guildsoftware.com/products.html
Erlang is a good fit for:
* Irregular concurrency: Task-level, Fine-grained parallelism
* Network servers
* Distributed systems
* Middleware: Parallel databases, Message Queue servers
* Soft Realtime / Embedded applications
* Monitoring, control and testing tools
Not so good fit for:
* Concurrency more appropriate to synchronized parallel execution: Data Parallelism
* Floating-point intensive code (HPC)
* Text Processing / Unicode (Unicode support now much better)
* Traditional GUI (some improvement with wx now)
* Hard Realtime applications
* Extensive interop with other languages/VMs (some improvement here with NIFs and Erjang - Erlang on JVM)
* No Windows 64-bit VM
I using Erlang for the things listed under "not so fit", because it's advantages out-weight some incomvinience.
Erlang was a good fit for us, mostly because of the tools available around it, such as Nitrogen http://nitrogenproject.com a powerful JavaScript/Erlang framework. Erlang apps seem robust and lean.
Iterate: http://github.com/zaphar/iterate an agile user story tracking app. Which is mostly useable but still in development. Used nitrogen which I found to be a really good web framework for dynamic applications.
A little homemade web framework. A blog compiler to generate a static site. A one-on-one chat client to be integrated with other already existing websites, a browser-based version of the Apples to Apples game, a brainfuck interpreter, maze solver, etc.
High number of processes distributed across multiple nodes with supervisor trees taking care of them all has worked well for us.
- a distributer crawler that works with python. from a typical rdbms approach that brought down creating inverted index's from more than an hour to few seconds. the euroka moment was when we stopped porting/writing for another language, sequentially, for-loop style approach to a more - processes and message passing. big 'why didnt i do this moment' a few years ago!
- a website , dashboard for the ad/content network using yaws on the LYME stack
- and btw - all that rumour about unicode is just bs. actually it was a blessing for us that content aka strings are just integers which when converted to binaries not only takes less space, pattern matches like a dream but all of a sudden your sytem can handle any language because to it - it's all just integers, english or swahili. that was a huge bonus even when getting investors, new untapped market opp's.
- an ad network, ( content network i guess without saying)
- a fire-and-froget thumbnail/screenshot system with queues & talking to imagemagick . i love writing and fire-and-forget tools, and with erlang, its so darn easy to write proceses's that have a life of their own. sure at some point you need to compromise on erlang's unique ability to handle RAM vs giving that up to research on how to handle large mnesia tables ( mnesia = inbuild db that has a limit on size of tables) . but it's worth the tradeoff.
case in point - our idea of scaling was...'reducing nodes' ie increasing efficiency
-2008 a million hovers in the year, 4 nodes
-2009 a million hovers every month, 3 nodes
-2010 a million hovers every week, 2 nodes i could bring it down to 1 node for doing the tasks as well, on just one 1 8gb RAM node.
the idea of reducing your costs, means that much lesser to profitability. ( we're now profitable, pretty much on auto-pilot on the backend, the 3-4 member team concentrating on increasing toplines,sales now )
in addition... - a wrapper to iui for nitrogen framework for building iphone webapps easily - wrappers to tokyocabinet ive contributed to called medici - remember writing wrappers to other things like rrd, aws utils, etc
all of the above have been used at hover.in, more at http://slideshare.net/bosky101
there's nothing i can think of that i wont/cant do in erlang anymore.
~B @hoverin