What works in other systems (intercom, for example) is allowing callers to pass a user ID with every call. Then you can match on that and unsubscribe the user rather than the email itself.
We can solve any problem by introducing an extra level of indirection