Memcached turns 10 years old today | Hacker News Reader

Memcached turns 10 years old today

arstechnica.com

151 pointsAndrewDucker42 comments

Brad Fitzpatrick is one of the most interesting programmers to watch. He's written a dozen really interesting and useful pieces of code. Whatever he's working on is probably worth your attention. Right now he's working on Go at Google.

https://github.com/bradfitz

https://twitter.com/bradfitz

Another good link is Brad's (old) company Danga Interactive - http://danga.com

Yes, another of his inventions (or something he was at least involved in implementing) is Gearman (http://gearman.org/), a queue/worker system which I used in my last job at a domain registrar/web host to hack up parallelised domain name look-ups. Similar to memcached, it was ridiculously quick and easy to set up and use, and bought me massive efficiency gains with very little implementation cost.

Whoa, he did gearman? That's awesome. I had no idea.

He also invented and prototyped OpenID.

He really is! He will also be on stage in Paris in 2 weeks at my http://dotscale.eu conference ;-) Come and meet him!

I followed Brad on LJ around this time, a time before I learned to program but was into running Linux/FreeBSD on my desktop and taking multiple unsuccessful stabs at building things. I remember his post about memcached:

http://brad.livejournal.com/1893677.html

I didn't really know what he was talking about but it was really fascinating and inspiring (as a lot of his tech posts were).

It's interesting to see that the idea came to him while taking a shower, similar to Clifford Stoll's "Operation Showerhead" idea that took down Markus Hess. I wonder how many of the world's most important inventions were thought up in a shower...

This conversation came up within the last week, though I forget where: Rather than being "thought up" in the shower, the shower is more likely the place where the idea seeps into the conscious mind. We're generally in the shower after a night's sleep. During that sleep, we're working through thoughts, ideas, problems. Before we can make use of that processing work, we need it to be accessible to the waking mind. The shower is a place that wakening happens frequently.

Sometimes it's a case of ideas popping into our heads when we've relaxed our minds from the problem. eg Some of my best ideas come to my while I'm on the toilet because I can be staring at a piece of code for hours and not find the fault. But the act of walking away from the problem and relaxing my brain for a few minutes is sometimes all that's needed to see the problem from another angle (and thus stumble onto the solution).

It's the same reason that I used to have a bath whenever I'd get stuck on a college assignment.

Absolutely! Taking 5 minutes away from a problem can bring to light the solution when you're not trying to "bruteforce" the solution in your head.

Same as sometimes when driving as I'm not focused on work can bring about good ideas.

I used to have good ideas on the toilet, before smartphones were invented.

Memcached actually turns 10 tomorrow - Wed 22nd.

I consider memcached to be one of the most important programs of recent time, it has been a core component for almost every large scale website you can think of. Ubiquity isn't always synonymous with quality, but in this case it clearly is.

Its incredibly powerful and very easy to deploy. A great piece of work, one that has saved my sanity many times, and congratulations to all involved.

I miss LiveJournal so much. I was so reluctant to leave it but with all my irl friends migrating to Friendster and then myspace I had no choice. Privacy settings were easy, friend management was easy, desktop client was easy (and powerful). RIP

-quasidan

I agree with this a lot. LJ by its nature encouraged writing, not just status updates. It was a much more interesting - if sometimes drama-filled - read than my Facebook news feed is with many of the same people.

Also, LJ groups have not been successfully replicated by any of the other major social media networks.

Reddit?

LJ by its nature encouraged writing

Wordpress does the same, which is why I primarily use it and not the innumerable alternatives.

I think the desire to do long-form writing, especially regularly, is a relatively rare one, which is why we see most people migrating to sites that mostly do images / sound bites / etc.

But the people who do do long-form writing will probably always be with us and probably always need tools appropriate to that genre. I don't know much about the history of LJ or why /how Wordpress supplanted it for long-form writers, but I do know that WP works pretty well for me, or at least well enough that I haven't seen any alternatives good enough to consider switching.

It is still around! Except for the fact 90% of my friends have abandoned it for less-featured platforms, it works great.

I'm always impressed how such obvious tools (e.g. memcached, junit) can change the face of software development.

Do you have similar other obvious ones in mind? (please no rails :-)

Maybe not so much a tool, but the AJAX paradigm and XMLHttpRequest is, imo one of the bigger developments in the Internet era.

And, Thank you very much for making memcached, and happy birthday! I've used this as a core part of my stack for years and love it.

edit: clarity

Git, or in general distributed VC.

Not obvious to implement. But the concept was obvious.

Even if DVC lacked the D, the concept of versions being a DAG was an incredibly important. A very big improvement to how classical CVS systems have developers think about history.

Wikis. Who woulda thunk that a website that anybody can edit would actually work, let alone become an important component of the web and most corporate intranets.

HTTP. The protocol is not that complicated, but it's become ubiquitous, and a cornerstone of the web.

Social news. Nowadays everybody takes links you can vote on as a "duh" feature and wonder what the big deal behind Reddit is, but when it came out in 2005, it was very much a "Why has nobody thought of this before? Oh, right, because nobody will ever use it" invention.

>Nowadays everybody takes links you can vote on as a "duh" feature and wonder what the big deal behind Reddit is, but when it came out in 2005, it was very much a "Why has nobody thought of this before?

It had been thought of before, and done before. The late 90s had tons of sites like that. When reddit came out in 2005, it was very much a "what is so special about slashdot clone #437?".

Didn't slashdot have you vote on the comments and not the stories, and the stories were still picked by editors? I think that was what was new about Reddit and Digg...they were just user-submitted stories and nothing else.

Yes, but everyone had told them over and over again to change that, they just wanted to maintain control. It certainly wasn't a new invention of diggs, I just used slashdot as the example of reddit being a clone because slashdot was the most well known. kuro5hin was(is?) exactly digg/reddit for example. Social bookmarking sites were around before slashdot too.

redis?

Apache Httpd

I must say, the best thing to happen to memcached since memcached has been (in my experience) twemproxy[1].

We ($dayjob) have been using it in production and it has been _solid_. twemproxy is quality engineering.

[1]: https://github.com/twitter/twemproxy

The other thing that nobody seems to have brought up is the forks, descendants and similar pieces of software it inspired in one way or another: e.g. memcachedb, Tokyo Cabinet, Redis, etc. Memcached's release was very much a watershed moment.

Here is the canonical presentation about Brad's fun with scaling Livejournal (including memcached). It's still a good read even now if you want an introduction to the whole topic:

http://www.danga.com/words/2005_oscon/oscon-2005.pdf

edit: Think about the number of decade-old pieces of software still used almost universally because they're still one of the most effective solutions in their field. That is what makes me appreciate memcached most of all.

Apparently the first version was in Perl - did Brad himself rewrite it?

I wrote most of the initial C server. Brad helped with code reviews, some bug-hunting, and IIRC he wrote the first version of the slab allocator. Prior to using the slab allocator, we tried straightforward malloc (thrashed under load due to fragmentation) and Judy arrays (didn't scale).

Brad handled the Perl client (initially the only client), and I think we hashed out the protocol together. I was an employee at LiveJournal at the time.

(I want to emphasize that the credit for the _idea_ of memcached belongs 100% to Brad)

Anatoly is too modest to say it, but he deserves credit for being the author of memcached as people know it today. Which isn't to say Brad stole the credit or anything like that, but rather to note most successful projects involve more than one person.

Thanks for the response. This is a very enjoyable read BTW:

https://github.com/memcached/memcached/commit/d4b68a8eca39e6...

Memcached is the ancestor of the front end for Couchbase, so while we regularly come in as one of the most consistently fast NoSQL databases, we can't take complete credit for that (other than picking a great foundation).

Such a great, stable, and easy to use product. Keep it simple and reliable.

Btw, did anyone else too think of Gimme the cash! when seeing this story http://www.youtube.com/watch?v=aY_aOtFAcO4

Kudos for creating and maintaining one of the most important pieces of software used on the Web today!

This makes me feel old.

Now I feel old. :(

Reply on news.ycombinator.com