Some notes on making a random Erlang program (egitd) faster

Actually I learned this myself just this week. Don't fear the binaries in Erlang. You need to learn how they work, and in particular they can surprise you a bit if you don't turn fullsweep on by hanging around in the binary heap longer than you'd expect after all processes are done with it, but you are not doing the runtime any favors by using a lot of "strings" instead. (Fullsweep isn't the only answer either, it's just an easy thing to quickly say in an HN post.)

In Erlang, slices of binaries are by default references to the original, and if you want a true copy (within a single Erlang OS process, sending it across processes will make one automatically of course) you need to ask for it, with binary:copy, available only in relatively recent releases. In the context of where Erlang tends to run, I think this is the correct default behavior, but it does mean that you end up having to do at least a little smidge of memory-management-type thinking. Since your Erlang code is very likely to be something that takes a packet in, does "something", and sends it somewhere else without hanging on to it indefinitely, this can work well as long as you don't leak references.

For those who didn't notice - this series currently has 5 more articles - you need to click "Next post" at the bottom. Wasn't immediately obvious for me.

I wonder what Github thinks about the changes to this program -- whether it's fast enough for their needs. I tweeted this at github; I hope they respond. http://twitter.com/zckzck/status/36447074698399744

Mailing list post announcing the blog series:

http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:56379:201102:p...

This was really cool. It just goes to show you what an experienced programmer with the proper understanding of the tools can accomplish. He right away zeroed in on the blocking nature of certain operations and "erlangized" it, making the entire thing almost as fast as git-daemon.