I recently found https://commoncrawl.org/ and started working on my own personal search engine. I'm surprised how easy it is to actually get decent results with just metadata that show up nowhere in Google/DDG.
marginalia_nu, the author of another search engine, is looking for ways to cooperate.