Hacker News Reader

Comment by creamynebula | Hacker News Reader

My search results were a lot better in 2006 when, I assume, they didn't have all these ML pipelines...

My view of it is that they basically lost the SEO spam wars. Without the ML pipelines, the top results would all be dominated by the smaller, highly skilled, SEO manipulators. They didn't find a way to cleanly excise that spam, so they resorted to a very imperfect hammer...giving a lot of SEO weight to large corporate entities. So basically, a different kind of spam dominates now.

Minor rant: I can't seem to contact an actual towing company directly, it's usually some external entity that ranks top in Google then they charge you more to find the actual local towing company. It says "Towing company in city name" but it's not local.

This was my experience last year when I wanted to find a locksmith. EVERY result on GMaps that was shown as being in my city was, in fact, some centralized company that seemed to contract out to guys working out of their cars. Every one took my name and said they'd get back to me (which they did).

This is certainly a problem with the locksmith companies, but I think there's also a Maps problem, too: Google enables this kind of commerce, especially businesses that apparently have a physical location but explicitly stated on the phone that I can't stop by there. It reminded me of all of the thousands of Delaware corps housed in a single building.

If Google made it so that you're only listed on the map if you actually have a physical commerce location, that would help a lot -- at least for those kinds of businesses that need it. Towing companies and the like may be an exception?

Oh locksmiths are a fun one.

The type of people get into locksmithing are like those who do computer security for fun - they like to figure out how things work and how they can be broken. Which means that every locksmith wants to figure out how to game the system. And the first thing that they all figured out is that people tend to select whatever locksmith is closest. So they all went and pretended to be in a million places around the neighborhood in hopes that THEY would get selected.

That was the case a decade ago. And it was a nightmare. Glancing at a locksmith search now, Google somehow cleaned that up a lot. But it doesn't surprise me that whoever is on top now is someone who figured out how to game the current system.

Does Google get paid more if they link you directly or if they link you to the middleman?

> The goals of the advertising business model do not always correspond to providing quality search to users. - Backrub paper

> This was my experience last year when I wanted to find a locksmith. EVERY result on GMaps that was shown as being in my city was, in fact, some centralized company that seemed to contract out to guys working out of their cars. Every one took my name and said they'd get back to me (which they did).

I had the same thing happen just last week! The locksmith who came for me gave me his personal phone number though and said if I needed help again, I could call him directly and he could give a better rate due to not needing to pay the centralized company a portion. Despite often having social anxiety with strangers, I've still found that just being friendly and treating people well (in this case still giving a tip alongside the cancellation fee after my super finally arrived just moments before the locksmith) ends up getting me better services than any research I've been able to do online. I ended up making a similar connection with the people we hired to install our curtain rods just after we moved in; I was talking to them about how the movers that brought my girlfriends things over to the apartment ended up damaging her desk in transit, and they said that they also do moving as well as installation/assembly, so next time we end up moving, we also will be able to hire people we know and trust (we ended up hiring them again for a few odd jobs since the first time, and we almost always end up getting one of the same three people we've met and made connections with).

Not sure what the problem is here? You require that a locksmith sits in a brick and mortar store waiting for phone calls? Their tools fit in a backpack, there's literally no need for a physical location. I guess unless you're getting keys copied but any hardware store can do that.

If it's a Google Maps hit, street view and look for an actual storefront? Many mom-and-pop locksmiths have stores with safes etc on display. Though that won't work so well with tow companies.

This is funny for me since I was in Maps in 2010-2011, and "fake locksmith addresses" were a problem even then.

The same is true for me for contractors of any sort. Top links are frequently referral farms.

For local search, I almost always use google maps instead. Maps search isn't perfect, but it beats web search hands down.

It's really annoying - you have to dig around, and many business do not have a web presence at all so unless you can find something local with ads, you find nothing.

Some tricks I've used in the past include local church bulletins (most parishes have some sort of a website and a weekly bulletin with advertising on the back), or local sport team sponsors, or local bars.

For towing, you can also try calling a local dealership or auto repair.

Or their only web presence at all is a facebook page. I don't have a facebook account so this used to be mildly annoying but lately I've noticed it won't let me see at the page at all. I click the link to visit the business's "website" and I'm greeted with a full-page prompt to sign in to facebook.

Yeah, their "local search" stuff seems to use yet a different set of criteria. You'll see similar complaints for other niches like locksmiths. Some set of SEO spammers has figured out how to use services like UPS-Store virtual addresses to fool Google.

There's a tonne of low hanging fruit google completely ignores, for example any page with an amazon referral link is almost certainly spam.

"But wait!", you say, "there are some legit reviewers out there." Yes, there sure are, by my starement is accurate, because for every legit review site with aws referrals, there are tens of thousands of ml created spam referral sites.

And so the real review sites are often lost in the mix regardless, which makss arguments to keep those results pointless.

But google leaves them there, and this is the same sort of site which, if it were an email, would immediately end up in a spam folder.

And beyond this, the other part of the problem is their ridiculous aliasing of search terms, which helps spammy sites come back as a response.

You say google lost? It's not losing, if you just don't care.

Frankly, it's just a return on investment thing. As long as only a few people per tens of thousands bolt, why spend the r&d?

> There's a tonne of low hanging fruit google completely ignores, for example any page with an amazon referral link is almost certainly spam.

This is the root of the problem. If Google targets the low-hanging fruit, the spammers will very quickly find a workaround. Google is trying - and failing - to use more sophisticated signals.

Spam is a virus. Google beneffited from the network effect, eradicating all opposition. But, like a European medieval monarch, it now has a poor immune system because of its lack of genetic diversity. The many, many SEO spammers are constantly experimenting, and they only have to find one flaw in Google's algorithm for it to win and spread quickly.

Google's monoculture cannot save us, however hard they try.

If you target the spam business model, it won't route around it.

But yeah, a healthy set of search engines will get us better results.

I put Amazon referral links on my technical blog whenever it's a hardware project. So far I have made < $5 on it so I should probably remove them. But for others I think it helps justify the time I spent writing up the blog and doing the project.

If there is a numbered list with referral links. 99% it's low effort referral spam trash. How they don't already filter this must be some sort active sabatague

Such links are relatively easy to hide from Google (obfuscate behind a redirect or inject with JS on user action), so if Google started using the links as a signal, spammers would hide the links, and invert the signal — only non-spammers who don't want to risk being delisted for cloaking would be left with the links, and penalized for them.

A different, more profitable kind of spam dominates now.

A kind of spam from which Google profits because it often embeds its ads and/or analytics.

Blog farms are way too popular now, too. People have figured out how Google recognizes "meaningful content", and they generate blog posts and pages with AI and ghost writers to pump out vaguely helpful articles.

They downranked pirated content results in 2018[1] so someone could just use pagerank and include those sites and have a better search engine for that sort of content. There's also the "controversial twiddler"[2] which makes Google only present mainstream opinions on various topics. I had to search yandex.com to even find a good article on it. There's some weird stuff on the Youtube blacklist, like the Las Vegas mass shooting from a few years ago that everyone forgot about that they never figured out the motive for. See the leaked "YouTube Blacklist" under the "Censorship" link in the project veritas article.[2]

[1]https://fossbytes.com/google-downranks-65000-torrent-sites-i...

[2]https://www.projectveritas.com/news/google-machine-learning-...

>smaller

Perhaps in other words, market forces (I'm thinking ROI) and economies of scale will win.

What I was talking about is that there was once a pretty large community of smaller SEO manipulators that did a lot of experimentation on what really worked. Especially in the "black hat" realm. Finding things like areas on big-brand sites that would accept user-generated content, comments, etc, where you could bypass some checks and insert links to sites.

They would experiment in a pretty deep way, varying things like the rate of new links, type of new links, variations of anchor text / bare links, and so on.

It USED to be very effective.

The larger entities don't really have to be that detailed. If you have that brand power, you can just cut partnership/cross-link deals and pay a little attention to things like anchor text in links, contextual text around the link, etc.

Edit: Ah, yeah, agreed. What got lost in all this was good content that had no big brand behind it. The indiscriminate hammer Google used to kill off small-guy SEO spam also pushed a lot of actually good stuff, stuff that never did any SEO at all, off the first page.

For sure. Way back any high Pagerank page linking to another would unequivocally boost the target site, whatever niche, but it's became more nuanced as you say.

What I was meaning was, the players who can manipulate the link graph most cost effectively tend to rank better and it favours those with deeper pockets. Not so bad for competitive niches and high volume terms, but muddies the waters for many other things.

That's because the web was better in 2006.

I think Google has made one change for the worse, though, which is strongly favoring more recent content. Increasingly, I think that change has been a big contributor to the decay of the web since.

Part of the problem is, I think, many searches strongly benefit from up to date content- programming tools, fashion, celebrities, things to do in X area, etc.

It seems that Google has decided that most people want the most updated information when they look for something, which I don't think is entirely unreasonable.

What I would love, however, is a way to turn that off for particular searches. Researching past events, as a trivial example, benefits far more from exact results rather than most recent tangentially related blogspam.

You mention options in passing, but it is to me the root of the problem: Google hates giving up control. Control means ad revenue. So we could have options that would make search extremely efficient for most users, but that would presumably be very hard to monetize in comparison. So we have no options, and everyone gets mediocre to bad results.

Since everyone I know in tech laments Google's decline into uselessness, I'm assuming this is not a sustainable strategy.

> everyone gets mediocre to bad results

Everything I've heard from people I know at Google suggests otherwise. Most searches for most people ... work. I too struggle to have google work in specific research cases, and I would like more power-user toggles, but basic searches like "$celeberty_name photos" or "$my_kids_school calendar" or "pizza places near me" just sorta work.

Hard to know anything for sure since results are "tailored". But Google used to be excellent for technical searches, whereas now it is unhelpful at the best of times. I'm guessing this isn't counted in "most searches gor most people".

> What I would love, however, is a way to turn that off for particular searches

But you can! In search results, click Tools and switch the Any time dropdown to Custom range... and you can specify a date range in the past. (Apparently, the custom option is hidden in the mobile version?!) I'm not sure how precise and dependable it is but it seems at least partially useful when I search for historical events.

I don't actually want to exclude new content, I just don't want to give it priority over older content if the older content is at least equally specific in matching my query.

I think you’ve forgot the websites with walls of white-on-white keywords/links at the bottom, trying to game pagerank. Many of these models were marked improvements compared to the generations before.

I don't understand how you're comparing the two.

You think the results were better in 2006, but the corpus was also different.

There was way less SEO back then. If they continued to use the old algorithm, your results would be worse than what you have now.

What are you saying?!

There was loads of SEO back in 2000 even! It brought alta vista to its knees, the number one search engine of the day.

Google got started, grew, because it filtered all that SEO spammy junk.

Do you have specific stats to back this up? Number of SEO pages vs good ones?

Or are you just presuming?

You're both right. There was loads back then but there was also way less than now.

Page rank wasn’t a pure algorithmic solution in 2006 either.

So, while the problem changed I am not sure if it got harder or Google’s priorities shifted.

Doubt it. Even today, obvious spammy content aggregators show up for many keywords with stuff taken directly from Reddit/Stackoverflow/Superuser/etc.

All of Google's fancy ML could've been replaced with a simple report button - enough people report a site and either trigger a manual review (best option) or just ban them (could be used against competitors... but negative SEO was already a thing and is still widely used).

> My search results were a lot better in 2006 when, I assume, they didn't have all these ML pipelines...

That's like an old person complaining that their body felt a lot better back in 2006 when, I assume, they didn't have to use their walker and glasses all the time...

Search is objectively worse today than it used to be, it's not just that "it's harder to use for old people", it's just worse.

>Search is objectively worse today than it used to be

How would you demonstrate that search is objectively worse? And how would you then show that it's a result of Google's algorithms, and not a consequence of the content of the Internet changing significantly?

>How would you demonstrate that search is objectively worse?

There's a few ways to do that. The easiest is to point out that almost any lucrative search query has -0- organic results above the page fold on a typical monitor today. It's all ads unless you scroll down.

Then, it's not proof, but how much time do you think Google spends on things that sit below the fold and aren't clicked on much? What would the financial incentive be?

> how much time do you think Google spends on things that sit below the fold and aren't clicked on much? What would the financial incentive be?

Quality organic search results is the reason they can have ads above the fold. There's tremendous financial incentive for them to care about that.

It's a result of people generating content solely to cater to Google's algorithms and accumulate ad and referral revenue. The internet changed significantly because of how Google indexes and ranks content. The proof is in the pudding... results for product searches are dominated by SEO referral link blogspam. Entire careers and businesses that didn't exist in 2006 have been built around this.

> How would you demonstrate that search is objectively worse?

How would you demonstrate that being 50 years old is worse than being 25 years old?

You ask people that are 50 years old or older because they have been on both sides.

> it's a result of Google's algorithms

Well, it's simply in front of your eyes: this [1] was not possible in 2006.

Anyway, the fact that you cannot easily find on Google why Google search results are worse, proves that Google search results are worse today than in the past.

https://www.theverge.com/tldr/2020/1/23/21078343/google-ad-d...

I was trying to be clever with my metaphor, but since a lot of people seemed to miss my point, I'll spell it out.

An old person's glasses and walker don't make their body feel worse. They're responses to an underlying change, and in fact make them feel better than they would without them.

Similarly, I'd argue that the ML pipelines and complexities in Google search aren't why search results are worse today. Rather, the web has changed with more SEO spam, walled gardens, content in videos, and search has changed in that you try to find more kinds of information than ever before. It's the underlying changes that make the search seem worse, and all of Google's fancy algorithms are imperfect responses to that. Without them, I'd be surprised if Google's results weren't far worse than in 2006.

The comment I responded to:

> My search results were a lot better in 2006 when, I assume, they didn't have all these ML pipelines...

made it seem like maybe the ML pipelines were somehow causing the decline in quality, rather than simply an imperfect response to changes in the web since then.

It would be cool if we had a snapshot of the web from the time you think it was better and could pit the algorithm of today against the algorithm that was contemporary with the snapshot. It would also be interesting to take the old algorithm and apply it to the web of today.

My bet would be that each algorithm would perform best on the web of its day.

I find that Google search results are still the best of any search engine for specific computer science and other specific tech-related topics, as long as you construct a fairly complex search string. However, for general information on things like world events, local news, national politics, etc. it's become little more than a mirror for corporate and state propaganda outlets. This is likely due to those very ML pipelines mentioned above:

Vince (authority/brand power), Panda (inbound link quality), Penguin (content quality)...

This represents a pretty severe narrowing of results on information and opinions - possibly the worst results are on Youtube searches for newsworthy events. It'd be very interesting to see what kind of content a pure PageRank algorithm-based search engine would generate today, and I'd be very interested in using such a search engine. Now, would it be overrun by SEO? I don't know, but it'd be worth finding out.

I kind of wonder if Google Scholar is purely PageRank or citation-count based, it still gives very useful results with relatively simple query strings.

> I find that Google search results are still the best of any search engine for specific computer science and other specific tech-related topics

even without Google account?

This is my Google experience from my temporary workplace in Barcelona

https://i.imgur.com/X2Avw2v.png

Then I click on "I agree" and whatever thing I search I receive Spanish results because I am in Spain.

Not my idea of "the best of any search engine", but ok...

I never log in, I find the key to better results is to keep adding terms to the query, and force the 'verbatim' option under tools. Restriction to specific domains (.edu etc.) with site: helps a lot, and the not option is sometimes helpful (-this -that).

Youtube search is awful. I once went through news queries on Google searches of Youtube (site:youtube.com) looking for popular independent media outlets (BreakingPoints for example) by -MSNBC, -FOX, -CNBC etc., and ended up with a string about 25 queries long, and those shows are just banned. They were feeding me unpopular corporate media shows with hardly any views or subscribers at that point, but zero access to independent media. Full-on propaganda manipulation.

> That's like an old person complaining that their body felt a lot better back in 2006

It's simply a fact that in 2006 Google search results were better.

Reasons might vary and could well not be Google's fault, but it's not old man yelling at clouds it's provably true.

Of course people that were not using Google before 2006 can't really know, just like people that are not old cannot experience how much better being young is, body functioning wise.

Your search results were a lot better in 2006 when there wasn't yet a huge population of sites cranking out SEOed garbage.

Google is trying, they're just losing the arms race of detecting crap content vs generating it.

Why not crowd-source it?

All Google needs is an obvious, one-click "spam" button for logged-in users. Clicking should add a site to the user's spam blacklist (which they should be able to review).

They know about what users search for and how those searches overlap and separate users into groups (not to mention all the individual details they have access to). When sites are marked as spam by enough different types of users, those sites can then be manually reviewed and their content blacklisted by Google preventing the same or very similar sites with a different URL from appearing.

Unfortunately, Google makes a lot of money from these ad/spam sites, so they have a perverse incentive to keep allowing them.

It is great effort to make a high ranking spam site and trivial to destroy it for Google. It would be a winning battle.

I think that Google just don't care and are happy to be a search motor for Reddit and Wikipedia and to answer questions like "How old is Lady Gaga?".

Man I have such a different search experience than everyone on HN. Google is amazing for me. Can someone give me a search query they think gives "objectively" bad results and maybe some links they would expect to see who up in the top that aren't there? Or is it that you search for something and you don't find anything? Or is it just that people don't like that there are ads?

I genuinely don't understand.

I feel the same way, I think one of the biggest startup opportunities now is to create a decent search engine. Google is absolute rubbish and a new player needs to step in and take away that market share that google is throwing away by means of a terrible user experience.

I strongly disagree that there's a startup opportunity in Google's relatively poor search performance.

Obsessive programmers like myself really want a search engine that does better and helps us find the weird corners of the web.

Most people do not care. They can type a question into Google and get an answer, and that's all they're looking for.

People love to say that Google has "relatively poor" performance.

Relative to what? Google has better performance than any other search engine. It's relatively poor compared to the imaginary ideal search engine that gives you exactly the result you want for any query regardless of whether the information even exists or not.

I think your startup plan should not be: "Step one: Displace Google Search." Instead, you find the underserved niche, people who care about quality search results (maybe in a specific domain), and you address that. As you conquer the small market you expand.

Google really does seem to be frustratingly bad these days.

And we have add a variety of competing search-engines that still aren't as good. It isn't like the idea hasn't been tried. For example DuckDuckGo, or even Bing, but there have been half-dozen others too.

Indeed. It's an arms race. Search engines versus SEO.

Wholeheartedly agree, but with an important caveat - Google is no longer fighting for the best interest of it's users, it's fighting for the best interests of it's advertisers. How useful the results are is now secondary to "how much can this search be monetized". The former was important in the early days when Google needed to be competitive on search.

Most directly, if the search results were perfect you would never need to click on an ad.

The most evil thing Google is doing now is pushing brands to buy ads on their own name so that they appear above the organic #1 search result. It's like the time Facebook decided it wouldn't send messages to followers who like you unless you paid up.

> The most evil thing Google is doing now is pushing brands to buy ads on their own name

It's not like this is a new thing. The court cases about it are almost 20 years old: https://en.m.wikipedia.org/wiki/Google_v._Louis_Vuitton

That doesn’t mean with present day’s internet content and the terms you search for in present day would have returned better results using google from 2006

I think advanced SEO tactics would destroy the results in 2006 though.

Ditto.

Reply on news.ycombinator.com