The important part is having physical access to the phone. A targeted attack against you now requires a physical element, rather than being entirely online.
Here’s an old story of a friend who had a weird talk with someone who had redirected their phone: