Redis: under the hood
pauladamsmith.com
pauladamsmith.com
1) The Server structure in the unstable branch is finally divided into sections using comments.
2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.
3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.
4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().
5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.
7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.
8) Response reading is now faster and cleaner code, thanks to Pieter's work.
This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...
In just 30k lines of code there is already a lot to take in mind!
Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.
- Did you win the Putnam? If not, please don't be `bolder' than this guy? - Yes, I did.
Epic!
What changed since this article was written:
1) The Server structure in the unstable branch is finally divided into sections using comments.
2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.
3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.
4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().
5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.
7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.
8) Response reading is now faster and cleaner code, thanks to Pieter's work.
This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...
In just 30k lines of code there is already a lot to take in mind!
Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.