As I've heard it explained, the server will push what it thinks is relevant [1] and if the resource is already cached in the client, the client will ask the server to stop the push.
The relevant mechanism seems to be described here: https://http2.github.io/http2-spec/#PushResponses
[1]: By some criteria. I don't have details here.