Yeah, I've observed the effect, but it never occured to me that it doesn't (or shouldn't) have to be that way :) We should be able to demand codecs that are smarter and more adaptive :)
It's hard to imagine why anyone would want a blocky highres lowbitrate stream if the lowres variants are always better, for a given timespan of video