One idea: try using adversarial training (i.e. train the classifier on examples with bounded perturbations which maximize error).