Redis, Memcached, Tokyo Tyrant and MySQL comparision

It's worth noting that the numbers for MySQL are slightly artificial. In the real world, you would use a relational database and ask it difficult questions that take a lot longer to answer than just getting key/value pairs.

If you just look at the numbers on that page, you would think that Memcached is only 10 times faster than MySQL, but in reality, the speed increase is closer to 100 times or more, because MySQL grows slower the more difficult questions you ask, and the more data you've stored in it. Memcached performance is (almost) independent of the size of the data, and it can only really do one thing, but it does it well.

Then again, if all you're looking to do is to store key/value-pairs, this benchmark is useful.

Given the fact that two of the options are persistent and two are in-memory the results seem hardly surprising. What is kind of interesting is that when you look at the results closely you can see what is really being measured - client interface speed. Simply put, there is no way that a tokyo tyrant hash database and a memcached instance should run at the same speed. The former is writing to disk while the latter is not. This means that the bottleneck here is not the cache being "compared", but the client interface code.

If I had the time I would rewrite the client code to use the pytyrant interface talking to an in-memory tyrant hash database to see where things really stack up.

EDIT: If anyone wants to run the benchmarks, just replace the import of memcache in tokyo.py with an import of pytyrant, then replace the run loop with:

    t = pytyrant.PyTyrant.open('10.90.15.96', 1978)
    hashval = md5(str(self.c) + str(i)).hexdigest()
    if self.gs == 'set':
        t[hashval] = hashval
    else:
        t[hashval]

Is memcache a substitute for MySQL? I thought memcache was meant to ease off the load on database by caching query results. it's not like comparing apples and oranges, is it?

Maybe I don't understand what this article is getting at, but isn't the idea of comparing a relational database, a no-sql database, a caching solution, and a half caching solution/half no-sql database (redis) fundamentally flawed?

It seems a lot like trying to compare apples and oranges.

The problem with many of these benchmarks is that they aren't exercising what they think they're exercising.

Notice how tokyo tyrant and memcached track each other on the performance graphs while redis is doing much better? That's because memcached and tokyo tyrant using the same client library. Clearly the memcached client he's using isn't as well written as the redis client he's using.

We in the memcached community are trying to figure out which clients behave properly and describe relative performance between clients so people who read things like this can get a feel for what that actually means in real world applications.

The MySQL behavior that was seen (long connect times of unauthenticated users) may happen because MySQL can't do a reverse lookup on the IP. Setting the private ip's in the /etc/hosts file on the MySQL box fixes this problem.

I enjoyed the article, and have circulated it internally, but It's a shame that he couldn't run MongoDB reliably- Redis versus MongoDB would have been the most interesting to me.

Loved the quote at the end. Go Marathi manoos!