Hacker News Reader

I agree. I'm particularly interested in the full text search module. A lot of libraries that manipulate data structures in Redis could benefit from being able to modify data as a Redis module.

View on HN

Author of that module here. I'm recovering from jet lag after flying back from redisconf, and hope to open it in a couple of days. Mostly cleanup and some documentation left.

FoA here - it is gonna be awesome #FTFTW :)

>Inverted indexes are stored on top of Redis strings using the modules DMA method.

What does 'DMA method' mean here?

I'll expand on it on the repo, but the idea is this: In redis modules, you can take a redis string key, allocate as much space as needed, and get Direct Memory Access to that key's data, with zero copy overhead. You can then resize that string and grow it.

So basically you can model data structures on top of this, provided that there is an easy way to convert offsets to pointers, or that your structs are scalar only.

In the case of the inverted index, it uses delta and varint encoding of the index records, so no real mapping to structs was needed, I just traverse the data and deserialize each entry on the fly in a pretty fast manner.

Another example was the use of this for bloom filters, that directly manipulate bits on the key.

Reply on news.ycombinator.com