Hacker News Reader

Comment by blutack | Hacker News Reader

Redis doesn't guarantee that expiry happens exactly at expiry time ([1] gives up to a 1 millisecond delay), so if your system assumes that all related keys are removed at exactly the same time it is still broken (although the race window is now tighter).

Maybe you could store the related keys in a hash or something?

I'm also guilty of trying to use features in redis in ways they weren't really designed for - it's such a fantastic bit of technology that a lot of the time you can still get away with it but it's not redis's fault when it doesn't work!

1: https://redis.io/commands/expire

View on HN

Right. Relying on atomic and absolutely reliable expiration is the problem anyway. If it's really that important then it needs to run in a transaction that manually sweeps the keys.

yep - part of our fix was ordering the expires such that the invariants are maintained even if the expires arn't exactly at the same time (we are assuming expires are at least ordered)

Redis expires can't be ordered, unfortunately. [0]

One hack I can think of is, accessing the keys. If the key is expired, then Redis won't provide it and you could know it is expired.

Another way is to club all of the relative keys in a hash [1]

[0] - https://redis.io/commands/expire#how-redis-expires-keys

[1] - https://redis.io/commands/hset

yep - as i mentioned in a comment below - we arn't concerned about when the key is actually deleted - only that its not returned when looked up if its TTL is hit

sorry should have been more clear

no need to say sorry! Thanks for responding, and the post was fun to read.

I’m not aware of the docs making any guarantees that expires are ordered either, just that at some point after the expiry time the keys will be expired.

sorry i should have explained. by ordered i meant after expiring a key its TTL will be set, which will never be a time before a previous expire. when it looks up the key, if its TTL has been reached it will not be returned (which we checked in the redis source). we weren't concerned about then the keys are actually removed

I'm pretty sure they aren't ordered. Redis active key expiry is a random process: https://redis.io/commands/expire#how-redis-expires-keys. TL;DR: Redis will randomly sample the keyspace and delete any expired keys it finds in the sample. If a large enough percentage of the sample was expired, it will repeat the process. Since the sample is random, it cannot be ordered.

There is also a "passive" key expiry where any read to a key will check the expiry time and delete the key if the current time is larger than the expiry time.

>we are assuming expires are at least ordered

Why are you assuming that? And if true, why do you think that assumption will hold in the future????? Redis could always tweak or change the behavior.

why are you assuming something like this? It's not guaranteed in the documentation, did you at least check that the implementation actually does it?

Reply on news.ycombinator.com