You should be able to revoke the leaver's shared key from the keyring and reencrypt the secrets?