I just don't pick up the phone to numbers that are not saved in my contacts anymore - seems to have saved me a tonne of hassle.