You are right, but first you ideally should perform scene change detection (to put keyframes at propper positions), and that alone takes quite some processing.
As such, they are suboptimal by default if a lot of motion occurs.
What H.264 encoder are you using that does not have a scene change detection option?