In the 2nd movie (and perhaps other places), he gets shot repeatedly with rifles. Even if the shots don't penetrate, the repeated impact alone would break ribs and leave him incapacitated.
Either that or the kevlar failing due to repeated shots. It's not meant to be shot in the same place more than once.
Still more believable than Matrix style bullet dodging, though.