Yes, these are “adversarial patches” in image classification, like https://arxiv.org/abs/1712.09665 . Similarly you can take these adversaries and add them to your own larger model, in an arms race.