Hacker News Reader

Comment by nocturnial | Hacker News Reader

> Let us start with the need for data, going straight to the point: one cannot build a competitive search without data collected from people.

Google had a very good search engine before they started to collect data from everyone, not after.

I don't have a metric to decide whether the search results from google got better or worse over the years. Subjectively and for me personally, they've gotten worse. And I'm not the only one who thinks this. I could be (and probably am) wrong, but I think the more personal data a search engine has, the worse the results.

When I think about it in that way, it makes senses why it has gotten worse. The search engine pays more attention to the data it think it has collected from me and how it relates to advertisers, etc... instead of paying attention to the actual words I wrote in the query.

View on HN

Google search results have definitely gotten worse over time. I can think of a few reasons.

* Search is now optimising to direct users to where Google wants users to go rather than where users want to go (i.e. to sell shit)

* Search is now heavily censored to be 'mainstream'

* Google disproportionately biases search towards ideologies and political leanings that Google prefers (remember that time they arbitrarily decided to remove all results for guns and inadvertently also made it impossible to search for water guns?)

Switching up to foreign search engines like baidu and yandex really opened my eyes to how heavily Google manipulates what I get to see.

Google search in the 2000s was significantly more useful than it is today, but most of it has nothing to do with collecting personal data and more to do with Google transitioning from a search company to a pseudo internet hegemony.

A better example of personal data collection leading to worse results is Youtube. Youtube puts people into weird tiny bubbles of content that the algorithm thinks will get viewing time from that particular individual, and prevents that individual from easily discovering vast swaths of other content they may find interesting. Anyone else have a Youtube home page full of videos from 3-10 years ago that they've already seen?

Google is starting to feel like the paperclip optimiser. I'm sure the way its products work is currently optimal for whatever metrics they set, but it's certainly not optimal for me as a user. Google makes me feel exploited.

Or Google simply could not keep up with the pace of web growth. The web is already 100x bigger than it used to be 10 years ago and the external environment is actively gaming with Google (SEO, Spam, etc...) while the information available from a user query is still effectively same. Personalization may add more implicit information to a query (e.g. If you're tech-savy, then a keyword "Rust" more likely means PL than game), but most of the time it lacks the actual context so its usefulness is somehow bounded. Most of the useful information still comes from a query itself. Hopefully, advancement in the field of NLP may allow a longer query to be more effective but I don't see any straightforward way to keep the search quality with a 1~3 words query.

> Google is starting to feel like the paperclip optimiser. I'm sure the way its products work is currently optimal for whatever metrics they set, but it's certainly not optimal for me as a user

This makes me think of Goodhart's Law (https://en.wikipedia.org/wiki/Goodhart%27s_law)

> "When a measure becomes a target, it ceases to be a good measure."[

[Disclaimer, work at Cliqz]

Comparisons of quality are relative, Google was good from day one because it was better than the competitors at that moment of time. Problem is that if one starts using X, and that provides an edge, the other have to follow just to keep up. A very unfortunate rabbit hole.

I'm with you 100% that personal data makes search worse in in many cases, too much personalization is detrimental.

One last thing, which I'd like to stress, is that data from users is not necessarily the same as personal data. For instance, you can use the data from users as if they were just sensors, without trying to build sessions out of it.

In a way it's similar to what Google made when they started. Google was the first and only to use anchor text, which is user generated and a less noisier description of the content of the page. (True that they did not use the users themselves but they did the proper automatic crawling). But in a way they collected sensor data from users, not personal data. That's what we should still be doing today. The problem is , however, however, is that is too easy to go beyond "sensor data" and start to collect full sessions and even personal data.

> I'm with you 100% that personal data makes search worse in in many cases, too much personalization is detrimental.

This isn't even my main objection.

Shouldn't data collection be reciprocal? I submit data you can use and you submit the data you extracted and used from me?

The ideal would be when I submit a search query to your server and it would respond "I have collected (this) data from you" "I have put for auction data.{xyz} on market Y" "The companies bidding were a,b,c and they offered amount g,h,i" "Company U won the bid, that's why you're seeing this ad".

If you think about it, there's no way someone would be transparent enough to give this data to the users.

If "you" aren't willing to share your "sensor" data, why should we be forced to share ours?

Probably lots of people remember the good old days of Google Advance Search. Or the ability to fine tune the results with special operators like +, -, site:, etc. It was probably 2010 when we started to talk about filter bubbles and how "smart" algorithms adjust the data to our liking. No one even though about disasters like Cambridge Analitica. As those days are long gone, there is perspective that is not purely negative. IMAO modern search engines are more approachable. They are easier than ever to use. Specifically for people that don't necessarily know how or have patience to learn how to use it. We are power users and often forget that most people don't have to be ones. They just want to search.

Therefore approach that Cliqz takes to its data collection makes even more sense. What Human Web does is extraction of common knowledge (or sense) on what does Web mean to people. Not to you, not to me, but to all of us.

Reply on news.ycombinator.com