You can't reduce first-visit-to-site-B latency below what it would be if you hadn't visited site A and also not reduce first-visit-to-site-B latency below what it would be if you hadn't visited site A; no possible delay policy will help with that.
> they can control the latency of non-cached requests.
This is more of a problem, but it doesn't need latency as such - site B could compare request traffic to b.example versus cdn.example to see whether a request was skipped due to already being cached.
(To be clear, I'm not sure you can actually do cross-origin caching securely for web traffic, for the above reason; I was addressing the narrow question of how to pick a delay - namely, don't delay cached and uncached responses by the same amount relative to their naive latency, because the whole point is that their naive latencies are different and we're trying to make that not true.)