Note on the specific intervals: that's where the clever optimization comes in: leave as-is (passthrough) all not affected sections of the video (basically everything) and only recode the tiny fraction that needs recoding due to the cut being set on a non-keyframe.

That one, I do believe should get more research and implementations. Currently I only know of LosslessCut's "Smart" mode, but I'd like to see other big players of the video editing space improving the state of the art in that direction for simple cuts.