It probably just makes wasps really angry.
For those, make sure they don't set up shop and build a nest. If you see them starting on one, scrape it off immediately. They'll even nest inside your house, wriggling through a hole, so observe where they go.
Insecticide that kills on contact is probably the best way to deal with an incursion.