Hacker News Reader

Comment by jacquesm | Hacker News Reader

One nice solution would be a search engine that throws out anything related to commerce or linking to commerce.

Not entirely throwing out commerce, but I sometimes use https://millionshort.com/ where you can remove up to the top 1 million websites from the results.

Jeez, that's surprisingly (and kind of tragically) effective!

lol, if this becomes popular then everybody needs to reverse their seo efforts and optimize for position 1 million instead.

Thank you, I already knew about it but had somehow forgotten.

Amazing! This does something that I wish they would all do — allows me to block a site.

I never, ever, for any reason, want to see results from the Daily Mail. I don’t even care if it’s the most relevant content: I just never want to see that site in my results. I know I can block it in hosts or whatever, but that’s not what I want. I don’t even want to see the link.

I wish DDG would give me this option.

I feel the same about sites like inc.com, fastcompany.com, entrepreneur.com, forbes.com. They are the absolute worst, shallowest content for business journalism, but they're right at the top of search results, and their blogspam fills up my Pocket "Discover" recommendations.

Hey, thanks. That's a great site! You get some very interesting search results and – even better – there's no third-party JavaScript or other trackers.

Then the search engine itself would be probably thrown away :-D

My favorite parts of the internet have no relation to ads or commerce at all.... including wikipedia. When you’re looking for information commerce is a massive distraction.

sadly true, but an open source engine like searx could be made to do such a thing

How could it be sustained? Search engines are expensive to run.

It could be a subscription service.

The real cost is in the infrastructure needed to run it unless you go full P2P

P2P it is then!

Yes, Search could use some Bittorrent-like disruption.

I wonder though - is too much of the P2P universe now mobile devices instead of regular computers that it wouldn’t be feasible?

P2P can be done on mobile but you're fighting against limited resources (storage, bandwidth, CPU). The reason P2P got good on PC is because storage got cheaper, bandwidth was increasing and CPUs kept getting faster.

Right. So texting on P2P is easy and appropriate, but I can’t see how building a search engine on P2P would really work well given those limited resources.

It's important enough, and currently wastes enough of my time (read: SEO crap, Medium and Reddit are top results for... everything) that I'd happily pay for a good search engine!

There's YaCy already: https://yacy.net

Agreed. Much like a spam-filter. Seems straightforward enough. What am I missing?

What you're missing is the fact that larch search engine companies have no profit motive to do this. They get the vast amount of their money from commercial ad sales -- ads for commercial results. Why would they purposefully castrate their highest paying customers?

And if you think the answer is to create and host a new, independent engine, you will be hard pressed to develop a good one, and to find the money to keep it running.

How about a layer over Google? Take Google's search results and filter them. Any obvious barriers to that?

They are pretty aggressive about detecting that and throwing up captchas.

see user smt88's idea above.

>Any obvious barriers to that?

Rate limits.[0]

One can't really build a robust and comprehensive (server-side) search engine on top of Google's search API. In other words, Google Inc isn't going to provide an API so powerful that one can build an "alternative to Google" with it.

On the other hand, if you're talking about a "client-side" filter instead of a new search engine, (e.g. a client-side webscraper that flips through the Google results pages), you'd run into multiple problems. E.g. random CAPTCHAS screwing up the automation script, and the difficulty of creating a "rules engine" (e.g. machine learning) on your local computer that recognizes blog articles with a commercial slant.

[0] https://www.google.com/search?q=google+search+api+rate+limit

see user smt88's idea above.

user smt88 offered a fabulous idea :

>Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site.

>It's theoretically possible to change the UI or analyze Wikipedia to make a pretty solid search engine powered by millions of person-years of curation.

>user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site.

Using wikipedia as a data source for a "search engine" does not fulfill gp's (jacquesm) request of "One nice solution would be a search engine that throws out anything related to commerce or linking to commerce."

For example, go to the wikipedia page for the film "Groundhog Day" and look at the external links: https://en.wikipedia.org/wiki/Groundhog_Day#External_links

- one of the external links points to IMDB.com -- a commercial website owned by Amazon.

- another link points to Punxsatawney Groundhog Club website that's advertising a $30 beer tasting event (Hogtoberfest).

- another link points to a souvenir shop

To continue the example of commercial links, a lot of math articles on wikipedia point to MathWorld which has advertising for the commercial Wolfram Mathematica software package.

IMO, I don't believe a search engine based on wikipedia's list of external sites (even with ignoring the commercial links) is going to be useful for the mainstream audience. I probably do 50+ searches on google every single day and maybe 1% might be answerable from a "wikipedia search". For example, I needed a DIY answer to disassemble a Moen faucet and Wikipedia has zero articles with those instructions. In contrast, the Google (and Youtube) search results has the information I was looking for even though it has the unwanted ads and "content marketing" blogs from plumbing brands.

Reply on news.ycombinator.com