Hacker News Reader

Comment by lamontcg | Hacker News Reader

> I was too young to use Google when it first got started, but according to many, in the beginning, Google had better or more accurate search results. One reason for this was the fact that most of the sites back then were indie websites, which did not care about SEO and money and created websites for sharing information.

> But I think all can agree that Google results reduced when people started to game the algorithm and create search engine optimized garbage websites.

Well not quite.

There were piles of SEO crap sites being produced in 2001. Google used to manage to react to any new exploit and improve their results. They actively fought that battle for at least a decade or so. They used to win.

It is only more recently that they seem to have given up.

View on HN

>It is only more recently that they seem to have given up.

They haven't given up; the OP has a point. The "sites" you are hoping for Google to return _don't exist_. Any website online right now that doesn't exist to drive ad revenue is exceedingly rare. In 2001, there were way more websites that existed just for fun; any tom, dick and harry could open up note pad and get a website online. That doesn't exist anymore.

It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists

No. There still are forums on lots of topics. But search how to fix X and instead of prioritizing a forum site with answers you get 5 top results trying to sell you the parts to fix X. The answers are always way down the list.

>There still are forums on lots of topics.

What is Reddit if not the largest forum in the world? You have the same gripe; you are looking for a web where people share things for free (like in forum); and when Google first existed most of the web is like that.

And if google were to prioritize forums; then SEO blackhats would do everything in their power to make every website look like a forum; and then you are back to square one. The problem is the profit motive, not Google's algorithms.

Yes but google crawls the internet it would be pretty trivial for it to scan a site and realize it was full of affiliate links and trying to sell things. They could rank those sites lower then just a site with text and not trying to sell everything.

Explain away how literally verbatim searching with quotes doesn't return verbatim results. Or how searches that match the very title of an article returns mountains of SEO spam before listing the article, if it shows up at all.

If there's an algorithm being gamed that causes that, that algorithm was designed explicitly to be gamed.

> It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists

That, and the Google search results are bad for the parts that do exist.

Well you are stating your opinion as fact, which is not useful when your opinion is wrong, which it is in this case.

Still lots of great content still out there, often times the target of my original Google query.

It does exist, just the same - probably more so, because the web as a whole is many orders of magnitude large now, so there's just more of everything.

However, 90% of everything is crap, so the good stuff is not buried under thousands, or millions, or mere billions of pages of crap, but trillions.

> any tom, dick and harry could open up note pad and get a website online.

This part hasn't changed one iota - you can still do _exactly_ that, even following a tutorial from 1996 to the letter, if you want. All that stuff still works fine - and I have websites that old, built that way, that are still up and work fine.

Google intentionally finds synonyms for obscure jargon that would have typically turned up these sites. The verbatim setting barely makes a difference.

The reliance on synonyms and semantically related topics has made iterative refining impossible as well, it's essentially non-converging.

My solution has been to search in multiple foreign languages, which looks less conducive to these "affordances".

YES exactly THIS.

When the "No Search results" page was deleted, the internet died.

Yeah, having spent a lot of time working with dense vector based techniques, especially in the context of a domain which is really similar to semantic search, I can say 100% that I don't want BERT and related techniques anywhere near my google search. That's the real crux of the problem, is the reliance on these sorts of models, and their utilization within query rewriting schemes.

> It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists

You ever try to search for a niche piece of software for your computer? Google surfaces the "Top 7 Best software" content farm articles before you ever get to the official sites. That's what some of us are frustrated about.

In this vein it used to be easy to find dissenting opinions on something:

<Topic> sucks

RIP "sucks".

Have you ever searched for a recipe in Google? Have you noticed that the top results always go on for a few pages of SEO-optimized storytelling and explanation before you actually see the recipe?

I'm sure someone at Google has. They could fix it if they wanted to, and boost well-known sites that get straight to the recipe (allrecipes.com, foodnetwork.com) instead of making you scroll through the SEO spam first. They haven't.

I used to think this too but if all the recipe websites start going straight to the recipe without the the personalization, then how would you rank them? Remember, there are thousands of these websites while there are only so many ways you can boil an egg, for example.

PageRank-like system, ideally based on real identities? (i.e. depending on how often the recipe is linked from Facebook, etc.)

Also if you are specifically doing it for recipes, just select manually websites that have precise ingredient amounts, very detailed and explicit recipes, that explain the reasoning behind each step, provide photos of how things should and shouldn't look like, why the proposed technique is optimal and/or the tradeoff of alternative techniques, the chemistry involved, comparison with other recipes, etc. (although in my experience most or all recipe sites are horrible and aren't even vaguely close to something like this)

Back in 2001 advertising was far more lucrative than it was today. So a website could be both: useful, and a for-profit, ad-supported enterprise. Hell, the reason why most online versions of newspapers were free was because advertising was so lucrative. Now, ads pay peanuts, and only make sense if you already have ludicrous scale, or are able to game search results with low-effort """content""" to get that scale.[0]

So it's both "less signal" (fewer high-quality websites) and "more noise" (more spammy SEO-to-oblivion sites).

[0] A similar thing happened to video advertising, which is why even really good YouTube videos are filled to the brim with ad integrations, merch tie-ins, and calls to like and subscribe.

Sure they do. It is just in 2001, there was 30M websites, and let's say 5% of them were good. And in 2020, there is 2000M websites... and the number of good ones is about the same (because there are only so many people willing to spend time making personal website). So the fraction is much lower now.

It is easier than ever to open your websites, and there are tons of free providers today. Look at any manual aggregator -- like hacker news or hackaday.com -- to see them. This is a variety I have not had in 2001!

Part of the problem is that Google started heavily favoring the age of a domain about a decade or so ago. If you spin up a new site, with high quality content, you simply won’t show up in Google for a long time.

Also, the amount of good content today is vastly larger than it was in 2001. Though you are right about the ratio changing.

It is largely Google's fault.

Example. If you searched the word "flight" in 2002, the first two results would be NASA links and the third would be the Museum of Flight. These pages still exist. But now the entire first page of Google search results are cost comparison websites. This is Google's badness, not the web's.

I mean, you do you but when I search flight I for one am looking to purchase a flight. If I want NASA links or the Museum of Flight I would specify that.

Alternatively, I could have searched for "flight comparisons" or "US airlines" back in 2002 if that's the type of result that I actually wanted.

The problem is that there is no way I can tell current Google to give me stuff that's interesting, novel and information dense anymore. It's all well and good if I knew that the Museum of Flight existed before my search, but the point of Google is to tell me about its existence. I can't tell Google "flight ... but no cost comparison or airliner website stuff"; such a query is currently not possible.

It's not possible because the demand for this is very low and implementation cost is extremely high. No one will pay for that.

> demand for this is very low and implementation cost is extremely high.

Demand is real as evidenced by everyone in this thread wanting such a search engine. We are not representative of genpop, but we aren't insignificant either.

Google had this in 2002 (i.e. a default to novel and info dense results unless otherwise specified), so it can't be that hard or expensive.

> No one will pay for that.

So we have ads.

> They haven't given up; the OP has a point. The "sites" you are hoping for Google to return _don't exist_.

I was beginning to wonder if this explanation was true but after starting to use Marginalia (mostly for fun) I now know the old web still exists - Google just doesn't let me see it.

Because its simply not possible. There are too many barriers now. You need to get a static ip for one. Domains cost more. Email for those domains costs quite a bit. And your friends are far less likely to visit since theyre all addicted to infinite scrolling websites

They have given up, as in if you Google anything technical these days you'll get tons of spam sites who just copies the content from GitHub and StackOverflow. Google used to punish sites that just copied content from other sites, but that isn't the case anymore.

I think the incentives are not there anymore.

Before the rise of smartphones a competitor could have emerged and take users from Google. Yahoo wasn't completely dead yet. Platforms were mostly open, a sea change could have happened (though highly unlikely).

Into the smartphone area, other engines became anecdotal (bing just exists in the background), there is enough lock-in to not have to fear against direct competition, and the main focus becomes having users spend more time on the web, search more, instead of staying in native apps. That's were, if my memory is correct, they ditch language based landing pages and start fazing out modifier support and all the "power user" part of their interface.

Fighting SEO scams becomes a waste of time when it has no real impact on wether user will continue to use the product or not.

When it came out, PageRank was a really innovative way to order search results that cut through most of the "SEO" at the time, which was webpages doing lots of unsophisticated things to range well for a given topic (e.g., putting that topic in the <title> many times).

Yeah. PageRank worked because web site owners were linking to each other as a service to the "Surfers", signaling what they think are high quality sites. There is an incentive to keep sites high quality or it'll risk being delinked.

When people stopped "surfing" and starting "searching", the whole mechanism broke down. Plus Google started using other metrics for ranking sites these days.

SEO before Google PageRank was basically submitting a site multiple times in an online form to Altavista and it would rank higher. As a teenager I automated this with the only programming language I knew, mIRC scripts.

The real experts or people who create great content just for the sake of sharing information never care about backlinks and whatever else Google requires for SEO.. OTOH the SEO scammers like Pinterest create little content and give the search engine exactly what it wants (backlinks, social signals, etc) which is why the good content is so hard to find.

But at the end of the day it's the search engine's job to separate the wheat from the chaff and google has been doing a very poor job of it lately. Like the article right said, the best is to prioritise websites that are about sharing knowledge not affiliate marketing, paid ads, marketing, etc.

Reply on news.ycombinator.com