They key would be hashed with the user’s details (ip address, value in a session cookie etc) so someone else can’t reuse it. Hell there are things like elliptic curves and DH which still seem magic to me.

Now sure if the identity provider and the site work together they could negate the anonymity, but given that for the identity provider anonymisation would be the key selling feature they wouldn’t want to risk that. Mullvad I’m sure would be trustworthy enough.