Hacker News Reader

Comment by revicon | Hacker News Reader

This runs afoul of the article’s warning to “ Never require a cache hit”

  It's important never to 
  require a cache hit - even
  as a soft requirement.
  Evictions can happen at
  inconvenient times - such
  as when some other part of
  the system is under load - 
  and there mustn't be any
  negative consequences to a
  cache miss

If the cache does empty unexpectedly and that is going to cause your infrastructure to die in a cascade, probably some kind of fallback will be needed.

View on HN

If you have a result that takes 10 seconds to calculate, and you need to read it faster than that, so you make a background processes to calculate it and keep it updated somewhere, you could argue that it's no longer caching.

It's an asynchronous read model or a projection and it's now just part of your application.

I find that when I'm in a dead-end conversation with people about caching that often they can't accept that pre-calculation and caching are two different concepts, and you conflate them at our peril.

For instance if I look up a value and pass it to three pure functions to ask about it, versus having the 3 functions look up the value and try to share a cache entry.

This is doubly wrong because the final question may see a cache eviction and answer a different question due to changes in the data. Now I may have a situation where True:False:True was not a corner case we cover because it's logically impossible, but it's happening in the code because the last one changed from False to True due to concurrent read and write.

The latter is also harder to test, because it requires mocks instead of manufactured inputs. And at this point the code is using the cache as global, mutable state, instead of semi-local state. Three strikes, you're out.

If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not go the DB or somebackup mechanism.

Caches in web architectures seem to be similar to NUMA style addressing and makes me think that cache invalidations in web should have failovers to an L2 or L3 if the L1 is gone differentiating between an expected invalidation or an unexpected invalidation (Redis down, let’s say). Sometimes in an emergency (security incident, let’s say) all cache items should be invalidated though resulting in said 500 or temporarily unavailable response while the cache is repopulated again and that’s something else that Djikstra would shake his fist angrily at from beyond the grave just the same.

I'd argue in that case, the cache is more of a local read-only DB replica :)

It's a subtle word change, but can communicate the slightly difference architecture and break the reader away from an assumption.

Yeah, I've had a similar thought to that before... as you scale up, the whole "cache" thing is just not a service you can have, because a cache must only speed up the thing being cached, it should ideally have no other visible impacts. As a site scales up and becomes dependent on a cache, it is really not a "cache" anymore, and both terminology and mindset should change as a result.

If your site comes crashing down because the cache went down, it's not a cache anymore and you should use a different mindset in approaching it.

The answer varies depending on a whole lot of relevant numbers that can vary widely, but an example of an answer I have in one of my systems is that I store all the cache queries more persistently than the cache answers. This particular system gets hammered with requests that are mostly the same as last time at certain times but spends a lot of time idle. The numbers work out that if I'm trying to compute the cache during one of my crunches, I need a huge amount of resources for just a couple of minutes, but if I restart the service between these crunches, there's plenty of time to recompute the entire cache before it is necessary with no additional resources. But that's just one particular example of not really being a "cache" anymore suitable in my very particular case.

Reply on news.ycombinator.com