If I continue with this thought exercise, a lot of the big indoor shopping malls around me have been knocked down and replaced with standalone outdoor stores (walled gardens?).
I'm not sure where things are going next.
1) The shift from spammy shit content as something to squash to something to allow and even promote over better content, provided it follows certain (Google's) rules. This shift (in terms of Google's behavior) happened ~2008-2010 and we haven't seen a period of spammy crap content getting heavily downranked since then, like we used to when they were still trying to stay ahead of it rather than give it a "legitimate" avenue as a method of control. Google's still being the most important search provider to appease has left the rest unable to direct behavior toward anything better than what does well on Google, so their results aren't much better.
2) A move away from actual or de facto open systems & protocols to deliberately carved up communities. The only thing keeping chat, Twitter-like services, and other social media—hell, even Youtube, so far as some kind of format for hosting video with metadata—from being standards or protocols is that business incentives reward "owning" a userbase (so you can better spy on them, and to keep anyone from providing a better, perhaps less-spying-laden client and "stealing" ad-viewing eyeballs)
Both of these are fundamentally problems of the spyvertising economy taking over the Web and I think a lot of the issues would go away if we could (legally—I don't think tech will do it) permanently and completely break that. More specifically a big part of the problem is Google, though of course the rest of the Web giants are gleefully following similar bad incentives.
One development that would be good for small web sites to look into is schema.org linked-data formats. Those might simply be too effort-intensive for the spammers to adopt (at a high level of detail) and perhaps too much of a commitment to quality and transparency (they would have to actively forge the info, which would leave them open to bans given the lack of plausible deniability), so they might become a viable signal of quality and lead to higher visibility in SERP.
(Similar for things like proper separation of style from content, that have always been advocated for in the web-standards community but are not really commercially viable.)
I'm not quite sure if others have experimented with this stuff already, but it seems worth trying.
However, I feel like it's moved more in the direction of being like broadcast television: lots of content that's designed to be consumed once and then forgotten. Maybe the television analogy oversimplifies the matter. Still, I think the more that content creators view their content as going into a permanent library, the better the quality.
A lot of web traffic today transits through applications and platforms. This isn't necessarily a bad thing; although I'd hate to see even more walled gardens. My hope is only that the small and independent web not go forgotten/ignored.
https://twitter.com/earthboundkid/status/1095385048008798208