It's sad that after 20 years of the web, it still doesn't work properly. These kinds of bugs should have been solved years ago.
Further, if you directly view an image and it breaks half way, the browser will hide it and say it's corrupt (and have no "show anyway" button), unless the size wasn't provided, then the browser just thinks it's valid, despite that it could parse the JPEG file and find out that lines of pixels are missing. Calling either of these cases a feature seems very biased.
We aren't even talking about one of the worse problems, which is when a page fails to load, the browser doesn't try to reload it - you have to come back and press reload yourself. I've wasted literally hundreds of hours doing this.
There's also this related bug which contains over a decade of discussion:
https://bugzilla.mozilla.org/show_bug.cgi?id=41924
I agree that it makes more sense to give the indication "here is an image but it couldn't be displayed" than to have them disappear completely.
Making it feasible for people to create websites quickly enough to be worthwhile entailed allowing the tag soup non-standard HTML we see in practice now, and the confluence of desire and technical limitations made plug-ins inevitable.
A "better-regulated" Internet wouldn't have had the Web at all. Remember that, according to the people using it circa 1988, file transfer and hypertext were both solved problems, with FTP and Gopher respectively already in existence. Unstructured hypertext, something Gopher was not really designed for, wasn't especially desired until some random at CERN released WorldWideWeb and httpd to the world.