If the problem was out-of-viewport images impeding the receipt of assets that do affect the viewport (CSS, fonts, etc.) then it would be good to postpone fetching those images just until everything else is received, rather than postponing until they're closer to the viewport.

Makes me wonder what other great ideas we've implemented.

Which is indeed why it's a terrible idea.