This is in fact exactly what we (https://www.turbinelabs.io) do. With enough proxy work you can tee some traffic to version Y and observe success rate/latency, then discard the response (or compare it with version X's response), at least for idempotent requests.

A sort of A/B test, where what is tested is the implementation.