I wonder if it would be possible to filter out the noise of the “modern web” by black listing big tech the same way we block ads. Might be an interesting project.
Maybe using an ad blocker to build a list of "websites that serve ads" (in addition to domains those ads are served from), and using that list to filter out results from an existing search engine API (like Duck Duck Go) would get you part of the way. But you'd probably need to search really far before you start finding sites that don't serve ads.
Not sure how useful it'd be, but I would use it!
It would probably be like Google circa 1999-2000
Edit - I think this was it, but looks like it needs an account: https://millionshort.com/
A lot of smaller sites just aren't indexed any more, it's a common complaint you read here...
"Smaller site, we don't use adwords, we're getting crawled 500x/day which is 40% of our bandwidth, still not indexed after 6 months"