You should not store any token in the frontend. Instead, have a server doing that and use a classic secure cookie session. The session is bound to the token / refresh token and can then do the token management.
Alternatively, don't use access token / refresh token but an ID token. The refresh of the token will be done via the ID token and the user's session (cookie) to the OAuth provider.