Based on previous decisions made by customers that were processed within the lifetime of the same request, there was some probability that the cache could be out of date.

Typically, there was enough context within the request to make assumptions about the state of the authoritative store if data hadn't propagated to the cache in time. We eventually moved to using that initially.

edit: I should add, though, the order of the datasources wasn't critical to the strategy: have a primary data source, backup data source, use any data available in the request context, and finally err in the customer's favor. That's 4 chances to get things right, none of which include retrying.