Hacker News Reader

Comment by seiferteric | Hacker News Reader

I have noticed in the past few years google results have become noticeable worse for similar reasons. Google used to _surprise_ me with how good it was able to understand what I was really looking for even when I put in vague terms. I remember being shocked on several occasions when putting in half remembered sentences, lyrics, expressions from something I had heard years ago and it being the first! result. I almost never have this experience anymore. Instead it seems to almost always return the "dumb" result, i.e. the things I was not looking for, even trying to avoid using clever search terms. It's almost like it is only doing basic word matching or something now. Also, usually the first page is all blogspam SEO garbage now.

View on HN

Google was good at launch because it was harvesting data from webrings and directories to provide it "high quality" link ranking data. However, they didn't thank or credit or share any of their revenue with the sites whose human curation helped their results become so impressive. Seeing that Google search was effective, most human curators stopped curating directories and webrings. The SEO industry picked up the slack and began curating "blogs" that are junk links to junk products. This pair of outcomes led to the gradual and ongoing decay of Google's result quality.

Google has not yet discovered how to automate "is this a quality link?" evaluation or not, since they can't tell the difference between "an amateur who's put in 20 years and just writes haphazardly" and "an SEO professional who uses Markov-generated text to juice links". They have started to select "human-curated" sources of knowledge to promote above search results, which has resulted in various instances of e.g. a political party's search results showing a parody image. They simply cannot evaluate trust without the data they initially harvested to make their billions, and without curation their algorithm will continue to fail.

> Google has not yet discovered how to automate "is this a quality link?"

Google has so much more data than just the keywords and searches people make, it seems like this should be a problem they could solve.

Through tracking cookies (e.g. Google Analytics) they should be able to follow a single user's session from start to finish, and they also should be able to 'rank' users in some vague way where they'd learn which users very rarely fall for ads or spend time on the sites that they know are BS. Those sites that are showing up on page 5 or 6 of the search results, but still get far more attention than others on the first few pages, could get ranked higher.

But I don't think many of Google's problems these days are technical in nature. They're caused by the MBAs now having more power at Google than the techies, and thus increasing revenue is more important than accuracy.

Theoretically, they could do a lot of things, but plenty of those would get them in hot water from a regulatory standpoint.

Also, don't underestimate the adversaries. Ranking well on Google means earning a lot of money. So much so, that I'd argue the SEO-people are making significantly more money than Google loses by having spammy SERPs. They will happily throw money at the problem and work around the filters. I don't think you can really select for quality by statistical measures. Google tried and massively threw "trust" at traditional media companies and "brands". The SEO-people responded by simply paying the media companies to host their content, and now they rank top 3, pay less than they did by buying links previously, and never get penalties.

Nope, all they could do there would be to group people together based on their "search behavior graph". The problem of finding BS sites is in itself a "shirt without stripes"-level hard problem. That's why being able to rely on user curation is (was?) so important for Google. People didn't curate the internet at first for money, they did it because they were interested in the subject for which they were curating their web ring.

SEO would just pay people to surf bad sites on the web, in order to feed the data into Google’s engine.

They already do this today for any venue where they can link “traffic volume” to “ranking increase without human review”.

I disagree with your first point but agree with your second. Google obsoleted most webrings/directories because page rank was a better way of calculating a websites popularity. Then, websites figured out how to game page rank, and its been a gradual decline ever since.

The point is that the web rings and directories were an important source of good PageRank input. By killing them off, Google basically clearcut all their resources and did nothing to replant, and now the web is becoming a desert where nothing can grow.

HN keeps mentioning dark SEO and companies gaming the ranking.

That might explain a lot but I don't think so.

Just look to how they are messing up simple searches because of basic lack of quality controls:

- Why doesn't doublequotes work anymore? Not because dark SEO vut because nobody cares.

- Same goes for the verbatim option.

- The last Android phone I liked was the Samsung SII, and last year I finally gave up and got the cheapest new iPhone I could get, an XR. My iPhone XR reliably does something my S3, S4, S7 Edge and at least one Samsung Note couldn't do: it just work as expected without unreasonable delays.

- Ads. They seem to be optimized to fleece advertisers for pay-per-views because a good number of the ads I've seen are ridiculous, especially given that I had reported those ads a number of times. I guess what certain customers that probably paid a lot for those impressions would say if they knew that I had specifically tried to opt out from those ads and weren't in the target group anyway.

Google was good because they used to optimise to giving the best search results, that was their aim. Now their aim is best profit, it seems, and their results appear to correspond (keep you on site longer).

An interview with the founders of Google from 1999 offers a more nuanced view on this. https://www.kalemm.com/words/googles-first-steps/

Google's aim was to replace other sources of information with Google:

> People make decisions based on information they find on the Web. So companies that are in-between people and their information are in a very powerful position

Profit was on their minds from the very beginning:

> There are a lot of benefits for us, aside from potential financial success.

Revenue, however, was not urgent back then, to them or to their VCs:

> Right now, we’re thinking about generating some revenue. We have a number of ways to doing that. One thing is we can put up some advertising.

So over the past two decades, they executed a two-pronged approach: Become indispensable and Become profitable. But now they're trying to pivot from "at web search" to "at assisting human beings", and that's a much more difficult problem when their approach to "Become profitable" was to use algorithms rather than human beings.

Here's a useful litmus test for whether Google has succeeded at that pivot:

If you were in a foreign city and you suddenly wanted to propose marriage to your partner, would you trust Google Assistant to help you find a ring, make a dinner reservation, and ensure that the staff support the mood you want (Quiet or Loud, Private or Public)?

If so, then Google's pivot has been successful.

Your search for "skiing Norway" mostly returns results for skiing in the French Alps, because those pages have much higher visit rates.

Google is a dumbass nowadays, and regularly ignores half your search terms to present you with absolutely irrelevant results, that have gotten lots of visits in the past.

I've noticed this too, and frequently wonder why there aren't new and better search startups...

There are, like DuckDuckGo. But the first complaint is usually "their results aren't as good as Google" and that's because Google in reality still gives better results because of their (lack of) privacy.

People want better results but don't want to be tracked, and those things are in opposition to each other.

IME DuckDuckGo has all the same problems people are talking about here. I uses DDG as my primary search engine.

It took me something like 6 years, but I've gone over to DDG. Their results were poorer than Google's for me, so when I tried to switch I used to end up repeating and adding !g to every search. I don't think DDG got better, but Google results are bad enough that I think they're equal in quality now (for me). I don't login to Google, have tracking disabled, use uBlock and pihole; FF/Brave.

> I don't login to Google, have tracking disabled, use uBlock and pihole; FF/Brave.

I mean, that's probably why they are equivalent for you. You've chosen privacy over better results (which is a totally legit choice to make!).

Well it's hard to tell objectively but it seems to have got worse without me changing the privacy settings. I guess that's their quid pro quo though.

I concur that both the UI/UX has gone down and the results themselves are feeling less reliable.

Have you tried viewing pages past the first page? Often times it's just filled with what looks like foreign hacker scam websites.

Yeah I guess it comes down to monetization strategy and how loyal/sympathetic the early adopters are.

It's funny because it's frequently mentioned how Google's tracking is what enables it to give such personalized search results, but often I question how effective that really is.

For instance I question if Google has some profile on me and shows results they _think_ I will want to see (e.g. news related), and thus leave out other results. If it works that way then I'm frequently seeing the same websites in my results and effectively being siloed and shielded from other results that I may find interesting.

Their new strategy of adding snippets for everything has truly gone insane. I search a query for "covid us deaths" today and had to scroll about 3 viewport lengths down to even see the first result.

What happened to just a plain list of blue links?

From a marketing perspective, I feel like DDG needs to change it's name or use a shortened alias. "Google" is an incredible word as it's easy to spell, remember, and it's short. Interestingly they own "duck.com"...

I think the only time Google still reliably gives better results than Duckduckgo is for non-English languges (which Qwant is good for), or when Google has something indexed and Bing/DDG does not.

But taking it as a given the Google's results are better, is that really because of lack of privacy, or just because of how Google has been pouring more money and talent into the problem longer than anyone else? Because I'm not convinced that personal data is particularly useful for generating search results. The example they always give is determining whether a search for "jaguar" means the cat or the car. But that always seemed silly to me, because most searches are going to give extra context to disambiguate ("jaguar habitat"), and even they don't, the user is smart enough to type "jaguar car" if they're not getting the right results. Further, Google doesn't actually know whether I'm more interested in cars or cats—it justs know that I'm a woman in college, so it guesses that I'm less interested in cars. Is that really so useful?

Does searching Google through Tor give noticeably worse results than searching google while logged in? I would be genuinely surprised if it did.

> There are, like DuckDuckGo. But the first complaint is usually "their results aren't as good as Google" and that's because Google in reality still gives better results because of their (lack of) privacy.

Alternative hypothesis: people only have had Google as reference for years, which means that Google represents "reality" to them. Anything that looks even slightly different is therefore worse.

I don't know about you, but 4 out of 5 times, when I research my ddg query, which did not produce a single relevant result on the first page, on Google, I get a good result as #1... Maybe I search google optimized.

Still though: This is not evidence for Google's search quality. I, too, feel, like the results got worse over the last years.

Also: Afaik, DDG uses bing under the hood, not what I would call "search startup" in the sense of revolutionizing search quality.

Is that actually true for you? I just tried it (logged out, and with adblock) and everything on the page seems relevant.

The results are correct for me. You just wanted to write a snarky comment. Did that make you feel better and gloat on Twitter about how you called Google a dumbass ?

I have also found that search results are getting frustratingly worse. Often even when I put in explicit search terms and quotes, and filter out words that I don't want, Google will return results that don't adhere to what I am looking for or just return no results at all. I remember when I would search for something and find much more relevant information. Now the first 5 or 6 search results are ad-sponsored and aren't relevant, but I have to go to the 3rd or 4th page to find something that matches. I also often have to search for things that were posted in the last year or less because the older postings are increasingly irrelevant.

Page 1: About 189,000 results (0.35 seconds)

Page 2: Page 2 of about 86 results (0.36 seconds)

It seems they're really just trimming the web.

I suspect their job has gotten way harder. It's easy to forget that they aren't just passively indexing. The web is basically google's adversary, with every page trying to be top ranked regardless of whether it "should" be.

I think it's disingenuous to say "the web is basically google's adversary" when Google AdWords is the reason so many pages fight for top ranking.

Well sure, but I was hoping my narrow meaning would be clear: the search team operates in an adversarial environment.

The Search team isn't some helpless independent group adrift on currents outside of their control.

If it wasn't AdWords, it'd be something else. People build websites because they want their content to be seen. That means competing for search result ranking.

That was when people built websites to deliver content. Now people build websites to get highly ranked in Google. No matter how good google's algos are, they can't win when the underlying content is just SEO'd garbage.

SEO'd garbage often contains ads, including Google ads. There's no incentive for Google to fix this problem.

Google's job is not to give you great search results, it's to keep you clicking on ads. Ideally it would be the ads on the search results page directly, but if that doesn't work then a blogspam website with Google ads is the next best thing.

If Google was a paid service this problem would be solved the next day. Oh, and Pinterest would completely disappear from Google too. :)

> If Google was a paid service this problem would be solved the next day.

Nope. Cable television was introduced with the promise of no ads. That didn't last long.

Spotify & YouTube Premium run without ads just fine, so why wouldn't it work for search engines? Text is way more compressible then music and video. So at least on serving data it is way more cheaper than the aforementioned services

Because an MBA will realize that they're "leaving money on the table" and not "maximizing shareholder value" by selling ad free subscription services.

Subscription based services also require you to be authenticated and that enables fine grained invasive tracking. Something traditional media couldn't do.

If delivery costs were a factor then I shouldn't be charged $15 for an ebook with near zero distribution costs when a paperback was $5 before ebooks came onto the scene and introduced a new incentive for price gouging.

Paperbacks were never $5 unless you were looking in the bargain bin.

Pulp novels were. Even in this millennium.

Cable television could get away with it because for a long time there was no alternative, so they could renege on the promise of no ads and still keep making money (though now people have alternatives in the form of streaming services and cable television is circling the drain).

Search engines are a relatively competitive market. A paid Google with no extra perks will not fly when the majority of people will just flee to Bing. For a paid Google to be successful it has to provide additional value such as filtering out ads, blogspam, Pinterest and other wastes of time.

I would gladly pay 7.50 euros a month for a search engine which would serve me good results without the SEO blogspam. Sadly, I think that no new pages are written without SEO bullcrap. The next best thing I think is to use the search functionality of sites themselves (like Stackoverflow, Reddit, HackerNews etc.)

Google has gotten worse for me BECAUSE of the stuff you're talking about: It used to search everything and find the words that I cared about.

Today, it will silently guess at what I want, and rewrite the query. If they have indexed pages that contain the words I put in, but don't meet their freshness/recency/goodness criteria, they will return OTHER pages with content that contains vaguely related words. "Oh, he couldn't have meant that, it's from 6 months ago, and it's niche!"

They'll even show this off by bolding the words I didn't want to search for.

So, if I'm looking for something that isn't popular -- duckduckgo it is. It doesn't do this kind of rewriting, so my queries still work.

I'm not sure what has changed but I've experienced of late DDG participating in a similar style of result padding/query ignoring which has been publicly brought up in Reddit feedback (and I've certainly complained about using their own feedback form). Seemingly arbitrarily even double quoted strings I've observed being ignored which is not how it was even six months ago.

I still continue to use it though since as some here have already mentioned Google's results because worse a few years ago and DDG was lean and good enough to switch. I do hope they'd consider more such feedback.

I agree with search getting worse, now I always have to add quotes around my keywords: yes, I really wanted to search for this exact thing. I also end up adding "reddit" in my query just so I can reach some genuine useful questions/responses instead of top 10 blog posts written just for the Amazon affiliate links.

Same! Or hackernews search when applicable, and sometimes even twitter is fruitful.

Imagine if somehow it would be possible to pick any release and search with that.

Reply on news.ycombinator.com