Hacker News Reader

Comment by klodolph | Hacker News Reader

That's a cat an mouse game that search engines and SEO companies have been playing for years. "Improve your benchmark" only works when you don't have adversaries.

Edit: To clarify, "improve your benchmark" is not a viable strategy by itself any more than "put your opponent in checkmate" is a viable strategy. As advice it is unhelpful. You generally have to mitigate flaws in the way you rank pages, keep large parts of the algorithm secret, and attack the economic viability of your adversary's strategies. Rather than going for checkmate, you want your opponent to resign so you can do something else instead.

There have been incentives for bad behavior on the web for so long that I don't think we have any hope of coming up with any kind of automated benchmark to tell us which pages have good user experience.

View on HN

> "Improve your benchmark" only works when you don't have adversaries.

If this were true, then there would exist no search engines consistently capable of finding content highly relevant to the user's query -- as opposed to just returning promoted content, which according to your premise is always winning the adversarial battle.

Sure, there is an adversarial element, which means there is a constant 'battle' for 'fairness' among results (or whatever your goals are as a search engine). But in reality, search engines as a whole seem to work rather well at finding what we want. Of course results are biased from adversarial pressures (and that's what we're discussing on this thread), but it does seem possible for "benchmarks" to work well in an adversarial environment.

Search engines work by not just improving the benchmark but also making the benchmark a trade secret. That's what I was saying. And if it's a trade secret I would say that the word "benchmark" is a misnomer. Broadly speaking, a benchmark is a standardized way of measuring something, and if only one company can perform the measurement then it's not really a standard.

> If this were true, then there would exist no search engines consistently capable of finding content highly relevant to the user's query -- as opposed to just returning promoted content, which according to your premise is always winning the adversarial battle.

I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do. Almost all my search results are dominated by low relevance high click through brands, regardless of quality. For example: quora often replaces high quality content with low quality paywalled answers researched from wikpedia or other public resources. 10 years ago you would have gotten metafilter, which is (imho) objectively a higher quality source less hostile to web users.

> I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do.

The degree to which search engines are gamed successfully by adversaries is not the variable I wish to present a strong opinion of here, which may explain your confusion on my position here.

Instead, I just wanted to illustrate the logical connection between an adversarial-defensive ability to rank based on [website loading speed], and the ability to rank sites based on [any other factors]. If you can pull off adversarial defense of the latter, you probably can achieve the former with even less effort.

That said, I do present to you tentatively the claim that most popular search engines do a decently good job at fulfilling their goals, even if those goals are not in alignment with users' goals in many cases.

For example, there are certainly many examples of paid ads given preferential treatment in search results; however I do not believe this constitutes a case of an adversary beating the search engine's metrics. Rather, it is a consensual relationship/contract between the search engine and an advertiser being fulfilled. Whether this contract/relationship is in the best interest of consumers, is another matter altogether, of course.

Reply on news.ycombinator.com