Comment by thamer | Hacker News Reader

Comment by thamer | Hacker News Reader

In practice there's a bit more to it than what the article describes, especially for a distributed cache: you need to have the Promise auto-expire so that if the machine that's performing the backend read disappears the other readers don't stay stuck forever waiting for it. It's also useful to have a feature in the cache itself that blocks the caller until the Promise has been fulfilled, so as to avoid repeated requests asking if the data is finally there.

As an aside, Guava loading caches[1] implement per-key locking so that multiple threads accessing a missing key simultaneously[2] would only lead to a single load with all other readers waiting for the loading thread to complete the fetch and populate the cache.

[1] https://github.com/google/guava/wiki/CachesExplained

[2] in the sense of "in the time it takes for the first accessor to complete the backend read"

> so that multiple threads accessing a missing key simultaneously[2] would only lead to a single load with all other readers waiting for the loading thread to complete

For anyone using Rails, Rails internal cache does this on expiration, but might not on initial load.

Guava cache's successor, Caffeine [1], handles the asynchronous case. In your scenario, you could set a timeout on the future as the entry will be discarded if the future results in an error.

[1] https://github.com/ben-manes/caffeine/wiki/Population

Why block the caller? Subsequent calls will have the same promise returned and notification will happen for all once the promise is resolved.

Because then your caller will need to implement something that understands what a promise is instead of just getting a data object it already has to understand. Not only this, but the caller will need to also implement a polling mechanism to keep trying. Put into code:

  func get_value(key):
    value = backend.get(key)
    return value

Is way better than something along these lines:

  func get_value(key):
    while true:
      response = backend.get(key)
      if response.type == "promise":
        sleep duration
        continue
      return response.value

If your service looks like the former, then suddenly your unit tests can use a postgres database, a sqlite database, a rest client... that all implement the same backend.get(key) interface.

Reply on news.ycombinator.com