This seems like the best solution. Introduce an opt-in security feature, whereby any attempt to port the number, or swap sims, is subject to a ~72h cooldown period. During that period, notify the account holder through numerous communication channels of the pending change.

Email, SMS, automated phone call.