Bullies are not "wild animals" bullying for "no reason", they're people with unique intellects and desires and motivations that you, or they, may not understand. They may want others to see them as tough, or to feel powerful or in control, or for laughs, or because they have something to gain, or because everyone else does it and they think it's normal.
Effective responses have to be targeted based on the motivation. I'm not saying you need to send bullies to the psychiatrist and "talk it out" to understand their motivations, mind you, just that you can't expect the right response to bully A to be the right response to bully B.
Sometimes the right response to a bully is to be a credible physical threat to him. Other times it may be to get to know him and make him feel respected, improving his self esteem, or to involve adults who will hand down more severe punishment, or to laugh it off, or to make fun of him, or for someone else to tell him "that's not cool". The right response might even be multi-tiered, trying to change his underlying motivations, increasing the cost of bullying, and decreasing the perceived benefits all at the same time.
Don't leave tools out of your arsenal by mistakenly thinking you should only "love on them" or only "fuck them up".