For the average internet user, small websites don't exist. Very few Instagram and TikTok profiles feature links to handcrafted sites. Google increasingly funnels all queries to the same 500 giant SEO'd sites.
Behemoths didn’t kill the open web. They created walled gardens which were more attractive to the 99% of people who aren’t interested in doing their own dev ops. Most people prefer to spend their time with their hobby rather than figure out how to set up a server or create ssh keys for GitHub pages.
I used Digg a lot for that, StumbleUpon was also really nice for this type of discovery, then early Reddit had a similar effect.
Nowadays? I don't know where to go, I can do all this effort of clicking around to find them but honestly I don't have the time, I'm in my mid-30s, I won't be jumping around hyperlinks searching for breadcrumbs of potential good content... A lot of people are doing that work already, like you, we are lacking a good place where we can pool this curation work collectively so others can discover it.
Though I did find https://cloudhiker.net/ recently, which is aiming for the same thing, and I'm optimistic.
I think that a problem for some people is that the straightforward solutions to discoverability, such as simply browsing the web in the manner you described or even what is given in the marginalia.nu article, do not solve the desire to be seen as urgently as more technologically coordinated processes.