Amazon has ruined search and Google is in on it | Hacker News Reader

Amazon has ruined search and Google is in on it

mdolon.com

293 pointsvincent_s282 comments

I'd like to note, donning my #2 at amzn hat again, that when the idea for affiliate sales originally came up, we certainly did imagine people creating pages with lists like "best bikes for under $500". However, I feel fairly confident in saying that nobody involved in that in the 95-96 timeframe was imagining that such pages would be created by anyone other than actual enthusiasts (probably a reflection of the state of the internet at the time).

In retrospect, this was a profound failure of our imaginations back then. It's also somewhat damning that in the 25+ years since, nothing about the affiliate sales concept has been substantially modified to mitigate the weaponization of this by "best X for 202X" pages.

Failure of imagination of the bad things Amazon would go on to do seems like the hallmark of the early days. Great job. Did you also game out the exploitation of warehouse workers?

I left Amazon after 14 months because, indeed, the exploitation of warehouse workers was obviously a part of the company's future. Didn't want to be a part of that.

An actual "best X for 202X" page is extremely useful; the problem is that there are no search signals to distinguish it from a trivially created spam mockery.

There's consumer advocate sites like https://www.which.co.uk/ that are pretty cool. I'd be interesting in knowing other trustable sources, mind.

I came here to say this. Embedding trusted sources like that [0] would a) help fund such useful organisations and b) help drown out link farms.

[0] in fact I think in Germany there is a state-sponsered impartial review body for consumer products.

Part of the point of my other comment in this thread [0] is that we shouldn't need to drown out link farms. Google chooses to not hide link farms. Does anyone seriously think that a company (Alphabet) that is capable of sucking in known protein structures and then spitting out predicted structures for thousands of other proteins is not also capable of identifying link farm pages? Yes, it's a non-trivial problem, but chess, go and protein folding are non-trivial problems too. Google appears to have chosen not to treat this as a problem worthy of its capabilities.

[0] https://news.ycombinator.com/item?id=27996639

There is Consumer Reports in the US: http://cr.org They are a nonprofit that has been independent for decades and have strict policies against accepting gifts, sponsorships, ads, etc, and take other steps to remain free from influence. For example, when they review new cars they send an undercover individual to buy it at normal price from a random dealer so the company and the dealer don't know the buyer is CR (and maybe sweeten the deal or throw in upgrades or take steps to improve its reliability or whatever). They also review a lot of household and kitchen appliances, mattresses, etc.

The New York Times now has their "Wirecutter" reviews of tech and household stuff, but they do rely on affiliate links for income so take that as you will.

There is not a single independent consumer organization that can test every product out there, so if you're looking at road bikes you may end up with a different set of trusted reviewers than if you're looking at computer hard drives or roof shingles.

CR also tends to limit itself to middle-of-the-range items. If you're actually interested in the best, their surveys don't always cover it, since they prefer to review things that most people can/would purchase. This has changed a bit in the last few years, but it's still the case that if you want to find solid unbiased reviews of, for example, high end ranges or washing machines, CR is not always the most helpful.

Good info. As a non-rich person I do appreciate their emphasis on highlighting the products that provide the best value for the money.

My dad was a big CR subscriber. After a while I realized that they focus on what might be termed "best value", although that's still not quite right. Their picks always emphasized mundane things like TCO, reliability, durability. For cars they were very explicit about this, each year issuing data on vehicles with categories like reliable, held their value, and low-maintenance, etc.

The prefect example of a CR-friendly car, in my mind, is the Toyota Camry. Completely unremarkable.

Everything else I remember them reviewing – watches, TVs, stereos, calculators, cameras – always ended up recommending some middle-of-the road "it works but it's not fancy and doesn't have many bells-and-whistles" product. I have a vague memory of them putting a Minolta camera at the top of the list of best 35mm SLRs in the 70s.

I found The Wirecutter quite useful. It's been purchased by the NYT, I hope they keep it going in the original form.

https://www.nytimes.com/wirecutter/

In Germany, there is the Stiftung Warentest, which has in depth tests of many household appliances and goods. According to Wikipedia, they cooperate with Which? in the UK and Consumers Union in the US.

https://www.test.de

https://en.wikipedia.org/wiki/Stiftung_Warentest

(For example, when they recently tested shaving blades, they had 23 testers shave their face half with one, half with a different shaver, randomised, then had each (half of each) shave assessed by the tester himself and an external expert blind (ie the tester did not know which shaver was used, or how the tester had judged the shave). Assessed where: quality of the shave; comfort; burning, reddening and irritations of the skin; cuts; how many shaves until the blades were blunt; ease of use & switching blades; cleaning of the blades; presence of polycyclic aromatic hydrocarbon and other chemicals in the handle; and more.)

I think that's a failure of imagination. In the early days of search engines and web crawling companies paid subject matter experts such as librarians and other experts to manually grade and categorize web pages.

Spam pages tend to be filled with Ads and derivative content, owned by spam company domains, and do not provide unique information relative to other sources.

If we as humans can recognize a spam page, why can't the machines? At the very least Google can penalize repeated spam domain owners/domains to help reduce the problem.

If we as humans can recognize a spam page, why can't the machines?

I think that many of these automated aggregators have come a long way. On the better sites, it's not immediately obvious that the page is spam, until I've read some way into it and notice patterns in the language and a lack of "meat".

My dad can't seem to recognize spam pages 99% of the time...

I find the advertising to content ratio is a decent clue as well.

Can humans recognize spam pages though? Back in the day, such pages would be created on blogspot or other spammy domains, so there was at least a visual signal of some sort. Now the spam is on Facebook groups, Whatsapp group chats, TikTok, Twitter, Medium, Linkedin.....the list goes on.

Online misinformation would not be as destructive as it is if people could tell the difference between say a credible news site and a Macedonian troll farm-run 'news' site written in broken English.

> Now the spam is on Facebook groups, Whatsapp group chats, TikTok, Twitter, Medium, Linkedin.....the list goes on

Of that list, Medium is the only one I would ever expect/want a search engine to even consider returning results for searches on "how to buy a bicycle for under $500". If I was King, you'd need to add allow:social to any google search to get any results at all from anything remotely like a social media or messaging application. I'd include LinkedIn and Pinterest and similar sites in that exclusion. If you want that stuff, you need to ask for it.

> Can humans recognize spam pages though?

Probably not, but they can recognize valuable non-spam. It's called by many names, but "reputation" is as good a term as any.

Yes, I'm saying that "Facebook groups, Whatsapp group chats, TikTok, Twitter, Medium, Linkedin" are all low reputation.

Ah, the old "Step one: create a general purpose artificial intelligence."

No general AI required. Simply count the number of ads in the text.

For a top ten site I’d also expect a certain degree of interesting vocabulary which loops in industry jargon. I’d also expect spam sites to use similar syntax/sentence structure for all products.

Lastly if I can follow the links and get references to the final products, then I can estimate how similar the product description is to the product. I’d expect most customers looking at top tens want something more than what they get from searching a retailers site.

Heck, many legit top lists include dedicated YouTube videos visually inspecting the products and reviewing them - cross referencing blogs to YouTube channels and validating that the YouTube video is of some production quality (likes/views/youtube Spam detection) could strip down a ton of spam.

> Simply count the number of ads in the text.

What is an ad?

Something served from double click or other major ad network

That does absolutely nothing to address the affiliate problem, nor does it catch advertorial or "native ads" as they sometimes called. Ignoring all content served by ad brokers is no more sufficient than browser ad blocking extensions. It helps, but you don't get an ad- and distraction-free result.

Plus, as soon as you make that a metric that is penalised by Google, how long do you think it'll take until ads are distributed from some rotating nondescript domains?

(See Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."

https://en.wikipedia.org/wiki/Goodhart's_law )

Don't spammers already do a lot of domain cycling? Doesn't NextDNS already ignore domains less than 30 days old, for exactly this reason?

Most affiliate spam sites that I see cover the page in intrusive advertisements to maximize revenue. Penalizing intrusive ads would certainly have some effect. This is also referring to penalizing rankings so that the entire page gets downranked rather than simply blocking ads in your browser.

It still wouldn't do much because as soon as Google starts to try to identify sites full of ads by some criteria, shady sites are going to start serving ads in some other way. It's already hard enough to distinguish ads in some sites that have gone with native ads or advertorials.

I don’t think there is a financial incentive to clean up the internet.

Too much money to be paid with shady practices that no one in power wants to do it.

Are those sites generated by bots now? They always seem to just be the top x results if you search on Amazon directly.

The reviews are usually somewhat accurate but only include items found on Amazon. So I'm guessing they hire people to write the reviews.

I have the impression that they use ghostwriters that summarise a couple of Amazon reviews, gain some superficial familiarity with the subject (talking points), but have no real experience with it. I've seen many of such sites (mattresses, bike accessories, VPNs, certain apps, etc.) that seem superficially plausible, but here and there betray profound ignorance of the subject matter.

Some for sure are (i.e. some really just are stringed-together sentences comparing numbers), many probably are just really low quality manual work.

I would guess most are written by humans. I know a lot of affiliates (mostly gambling but also finance and shopping) and most of their stuff is produced manually. There is some more automation these days but the industry is surprisingly manual.

I imagine that once these "businesses" get a hold of GTP2 or GPT3 they will crank them out.

I think they already have:

https://francescocolumbu.com/best-laptop-for-openbsd/

They're written by humans. I know someone who did this, she was paid by the word and would write "best 5 [product]" lists several times a month.

Obviously, you only have time to basically summarize points made on the listing and in the reviews for the product for it to be worthwhile.

This was pre GPT-2, though.

Google broke itself over a decade ago when they began giving you results either that their algorithms assumed you wanted or that they wanted you to have, period. Verbatim search hardly produces anything useful anymore, either. Search has been coopted, making the prime thing (past general connectivity) that made the internet so useful now all but worthless.

For this reason I started using Yandex. Few advertisers pay for non-Russian ads, and the results are better than DuckDuckGo (in my opinion).

I take your point, but don’t you think you’re overstating things a bit? The internet is still incredibly useful and so is Google search (granted, it takes more effort and sophistication to avoid the spam).

I don't think its overstated. The web is broken and search especially so. Now days I look for curated list to find interesting webpages and topics.

Google ruined itself years ago, or maybe "we" ruined it.

The original commenter said: "Search has been coopted, making the prime thing (past general connectivity) that made the internet so useful now all but worthless."

I agree that Google search has been coopted. It now requires a keener awareness of what is and isn't being artificially promoted. Is it annoying? Yes. Is it something that unsophisticated users fall prey to (by design)? Also sadly yes. But 'all but worthless'? That's a huge overstatement.

I don't think I'm overstating things. Yes, the internet is useful, but consider how the ability to accurately search is the real key to the whole of it. As the original post points out, this hits us in day-to-day matters just as much as in trying to find/discover useful research on a particular topic. The simple fact is this: not only must nearly every link you click be viewed with suspicion, so must every search result you're given by Google, and pretty much every other engine. What is most useful is a presentation of naked results (with functional boolean operators, exclusions that work, a lack of "suggested" terms, etc.), but that doesn't shape and cause more interactions, direct people to the "proper" sites, or improve the bottom line. I remember the internet when it was usable for much more than consumption; it is only marginally so today.

When you say "most useful", I have to ask, most useful to whom?

Agreed — Google search is most useful to Google. I'd prefer it be most useful for the actual user of the service, but that makes me crazy, I guess.

What if I told you that it is, but you’re not the target user? When you’re serving billions of users, there are going to be niche groups not worth catering to. I would bet most people are quite happy with the results.

Just like most people are happy with their experience at Apple Stores whereas I feel like they’re pandering and annoying because I know exactly what I’m looking for or what the problem is and they’re still asking me basic questions unintentionally wasting my time. I’m not the target demographic they’re catering to. It’s fine, I just deal with it.

You seem be saying there's a single "actual user", and that user happens to be identical to yourself.

In practice any product with more than a few users is going to have to separate out the different needs, situations, and personalities of the users. Personas and market segmentation are two of many ways to slice up the user base.

Some products can use that slicing to only address small niches and just say "to hell with everybody else". Indie games, for example, often address very specific audiences. Ditto a lot of books. Academic books especially might be written for an audience in the thousands or even hundreds.

But products like Google are going for extremely large audiences. Inevitably, needs will conflict. They can try to sort that out in product, and I think they've done a surprisingly good job of guessing what people actually want based on them typing a few words into a box. But when there's a conflict, it's no shock to me they're going to favor the vast majority of their audience over the relatively small niche of persnickety software developers like myself.

I’ve found DuckDuckGo can be better at some searches, and at others far worse. I was recently trying to search up how to do something in Excel and every result from DDG was some “excel-help” website that was actually just parroting common search terms as the headers on non-articles full of ads. The entire search space has apparently been hijacked by what is essentially a fake help site. I see results like that on google too sometimes but not across a whole results page.

These sites seem to have emerged in earnest over the last 5 years or so, designed to present the appearance of relevant content to the search engine but existing only to serve ads.

Most of DuckDuckGo's "10-pack" of traditional search results come from Bing: https://help.duckduckgo.com/results/sources/?redir=1#:~:text...

Whenever DDG is mentioned, these "but it uses Bing" comments always pop up. Sure maybe somebody is in the lucky 10k with TIL moment, however these "but it uses Bing" comments leave out the fact that Bing doesn't know you made the search. That's kind of the point in DDG is to be able to search the web anonymously.

I don't know the intent of the "but it uses Bing" posts to just be "in case you didn't know" or "it's lame because it just uses Bing" or something in between. It just seems to me these comments are missing the forest through the tree.

It speaks more to the quality of the results you can expect receiving non-personalized Bing results.

From my experience, it isn’t very good, and I’ve had DuckDuckGo as my default search engine for years, at first to give DuckDuckGo the old college try, but now primarily to be able to replicate a feature I had in Firefox in Safari and iOS Spotlight (being able to redirect a search to any search engine with a keyword).

DDG is fine for most things. It's made a lot inroads, not because its gotten better, but because Google has gotten worse.

Nah, most of my searches still start at DuckDuckGo since I haven’t fully gotten back in the habit of putting the search redirection keyword back in yet.

If it’s fine for you for most things, great, but based off very recent and nearly daily data on my end, that is not my experience at all.

I do agree about Google getting worse though. There isn’t a search engine today that can even hold a candle to Google from over 10 years ago including Google itself. Unfortunately, the closest thing is still Google.

I find DDG is generally good or at least good enough when the data you need is relative popular and relative well distributed around the internet. DDG falls very, very flat when it comes to needing to scour the depths of the internet for something very specific.

Case in point is car parts - find an obscure part from a Japanese Honda, for example, and attempt to find it in DDG. I'll wait. :P

It’s mentioned because Bing isn’t very good so you shouldn’t expect any different with DDG.

Also every time this comment is posted, someone tries to argue that it doesn’t come from Bing despite zero evidence so it’s a TIL for more than you think.

are people using DDG using it just because it's not Bing and/or G? I thought the whole premise of DDG was providing the ability to search anonymously by "proxying" your search to a different search engine altogther. I was aware of the switch away from G to Bing, and I was okay with that as now G no longer gets any search data from me even if it was made anon by DDG.

then use startpage, it takes its results from google

DDG is cool with the kids today, Bing is not, all matter of fashion.

Isn't the phenomenon you're describing (and I agree it's a real trend) more due to SEO efforts than to any defects in the search engine itself?

It seems like there are millions of how-to informational articles now that are inherently designed to be clickbait. It's not that they don't have useful information, but they make you dig through a lot more verbiage than is necessary, because apparently some search engines think longer = more informative.

¿Por que no los dos? Google has increasingly favored results that work in concert with the overall revenue-generating aspects of the web. Selling things has overtaken providing information, and SEO exists to push pages to the top of search results with the explicit purpose of making sites better at earning money. Google's business is selling ads and, perhaps more importantly, selling information about user preferences that allow businesses to better target potential buyers.

Just a random example I tried a bit ago. A search of "guitar tuner" gave results like Fender's online tuner site (stuff with plenty of upsells for Fender gear) a site that had a basic online tuner page – filled with ads, and a tutorial at the web site of a company that sells music lessons and runs music camps.

I think it is a failure on the part of the search engine if it falls for that SEO trickery.

I recall in the early 2000s it was popular to fill the bottom of a page with keywords in the same color as the background color, to trick the crawler into thinking the page was full of relevant content. After a little while that stopped working and would actually get your site penalized in ranking. The bad actors are never going to go away, they’re going to keep getting more clever and search engines like Google and Bing with billions in the corporate warchest ought to keep the edge in that arms race.

Similarly on Google and YouTube- nearly the entire search space for "how to pronounce xyz" has been taken over by videos of text to speech recordings that have absolutely no idea how to pronounce anything.

There are huge YouTube channels posting this scam content, and they never seem to go away. There are real ones in there too, but there are way more spam words than real ones.

Thank god for PronunciationManual, which seems to be the only channel taking things seriously.

https://www.youtube.com/watch?v=T6EekGFJhrg

I knew a student in a neighboring lab (very smart guy, postdocing at Stanford atm) who had to look up how to pronounce the word "schadenfreude". Sure enough he found the PronunciationManual video first, eventually leading to a hilarious conversation where his advisor was trying to figure out what he meant when he said "skoodenfruity".

I must say, PronunciationManual is one of the few YT channels that has me literally LOL frequently, and so did your story there. My enjoyment of those hilarious videos is just tempered a bit by pity with the innocent victims that fall for it.

FWIW, here's the link for Schadenfreude: https://www.youtube.com/watch?v=d3_DjiLLDfo

I never knew I could get pranked on HN. Kudos!

I use Forvo for this, it is great: https://forvo.com/

Seconded! I used Forvo for some Greek words and it had high quality recordings from actual humans with good accents.

I also had the same issue with these. So i started using these AI narrator platforms. These are generally used to convert ebooks into audio books but i used it for pronunciations and accents. I used the one called Narration Box. Here is the link- narrationbox.com

I've found DDG to be fine most of the time, but I wish they'd up their game on advanced operators. DDG seems to have a "feature" where it will ignore quotes around terms iff there were zero results for the original query, but not tell you that it did so. It's really annoying to get to the second or third page of results and realize that this happened.

These things have been around for a while. Back in the day, it was a major problem with yahoo. Every search you do would have 5 or 6 links to the likes of ask Jeeves which was in turn just showing you what ask would return if you put in the same search terms.

It's maddening.

Yep; I can’t believe what used to be bar-none the most useful set of search results will now seemingly scatter results into the list that have zero matches for my search term, as well as some with boldfaced words that I did not even type.

And also, for some reason the “zero-click results” stuff really bothers me. I don’t want some crappy AI summary of a Wikipedia page, I want the Wikipedia link first, for example. I can only assume this is a form of traffic theft because I seriously doubt Google has agreements with all the sites they slurp useful information from.

> I can only assume this is a form of traffic theft because I seriously doubt Google has agreements with all the sites they slurp useful information from.

Yup: https://theoutline.com/post/1399/how-google-ate-celebritynet...

It's possible in Chrome and Firefox (and I assume other browsers, though can't say for sure) to add search shortcuts for other websites. If I know I want the wikipedia link, rather than searching google for the thing, I type "wp <thing>", and it takes me either directly to the wikipedia page, or to the search results. I think the default is the domain followed by a space, so "wikipedia.org <thing>", but that can be changed.

this has always befuddled me. why use google at all to search a specific site rather than searching on the site directly? why give google the traffic? because lazy/convenience of google being the default page for new browsers and/or google the default search associated with the location bar?

Because search is hard. Most domain sites search titles and descriptions of articles for keywords. Google will index the entire page and be able to provide more and better ordered results.

Or use to. Now the quality and quantity of results forces site owners to implement their own search or use third parties.

This is still the case for plenty of websites. Searching google with "site:example.com search terms" often still gives better results than using a site's own search tool.

Most importantly, searching through Google generally gets me to my intended destination faster than loading some site and doing a search. Like you say, the URL bar in any browser I use also defaults to searching in the default search engine, so if it's Google, it's faster just to type it in there. So, from a new tab, I can just type in "wiki handball" to find the wikipedia page for handball and click to get what I want faster, instead of going to wikipedia.org, waiting for that page to load and then searching for handball and clicking.

DDG has the bang searches though, so typing in !wiki handball is faster still.

!w handball is enough (or !wde for the German version, !wes for Spanish, etc.)

Mediawiki's search sucks.

Unfortunately on some sites even generic google search (not limited to that site) is better than on-site search...

If you have DuckDuckGo in your browser search bar, then you get their bang functionality automatically, which replicates this. E.g. "bicycle !w" searches wikipedia. Includes !so (stackoverflow), !g (google), !yt (youtube), !a (amazon), !twitter, !reddit, many others.

https://duckduckgo.com/bang

For whatever reason, this feature is not actively promoted in Firefox (or Chrome). If you add a bookmark in Firefox, one of the fields that shows up is "Tags" which is sort of uninteresting. Then you have to go into your Bookmarks toolbar and find the bookmark you added to get to keywords so you can add a shortcut (my Wikipedia shortcut is just `w`). Then of course switch your search term with %s if you have a search shortcut vs just a shortcut to a particular page.

Chrome has an even more complicated way to hide this, although I don't regularly use it so I don't remember what the process was.

I agree that Firefox's method is a bit unintuitive, but Chrome's is dead simple. Settings > Manage Search Engines, and then just modify the ones it probably added automatically to have useful keywords.

Apparently it's bad UX to just tell you "we couldn't find any matches". You might leave the site and actually go elsewhere.

> And also, for some reason the “zero-click results” stuff really bothers me.

It's often inaccurate too. I've had a lot of issues with menus, hours, and phone numbers being completely incorrect when searching for businesses.

The one that really bothered me was when I needed to change flights and the reported number for an airline was just some fake support line scam.

It is always weird to me that people think the quality of Google results compared to the past are a deliberate choice by Google, and not the consequence of an adversarial process where marketers spend billions of dollars trying to distort the rankings

Google makes an annualized $71 billion in profit after paying tens of thousands of software engineers.

I’m not saying I think they should win outright but they should at least be holding their own if they are giving it their all.

They have a reputation for not motivating their best people to maintain and improve existing products. Maybe search is affected.

Also, it's a huge company. Maybe there are internal structures that when you work hard to advance your career within the company it doesn't align with advancing the overall company.

That had been the case for long. And google periodically reset their search algorithm purging billions of pages to clean up the junk. After which the results got better.

But sometime back, it stopped doing it. Or can't do it anymore.

Web-search is now up for disruption, when that happens.

Is it not a consequence of deliberate choices made by Google?

People would be doing this no matter what choices Google make because of how universally used it is. What choice do you think Google could make which would avoid that?

They could punish the rankings of review pages using affiliate links.

Clearly the use of these links creates a worse experience for users.

Not sure why you're carrying water for Google but I'm confident in their ability to filter obvious spam from their search results.

It's a bit of both. The company I work at has a pretty big SEO department, and it's definitely frustrating how they're able to game web search. But also last week I was trying to Google something about earbuds (I don't even remember what exactly), and searching for ("earbuds" -headphones) still brings up headphones as a bolded result because of how aggressively it tries to use synonyms.

Edit: just remembered a better example from earlier this week. Searching for "clock in a box paradox" gets results about schrodingers cat, or twin paradox articles. Searching for "clock in a box paradox -cat -schrodinger -twin" gets stuff about a video game called Cube Paradox. It's only when I search "clock in a box paradox einstein bohr" that you actually get results about Einstein's 1930 thought experiment.

> because of how aggressively it tries to use synonyms

That's a double edged sword.

One of the main reasons regular people fail to find stuff on Google, or at least failed in the past, was that you'd need to be a walking thesaurus to do it effectively (which techies generally are, for example).

They needed to help the less educated, too.

Bingo. In the past, you needed “Google-fu” to find what you were looking for. People that didn’t understand how the search was done under the hood (non-techies) would search with poor terms and get poor results. Nowadays, you can’t “game” the system with precise keywords but millions more people find what they’re looking for. Overall, it’s an improvement.

Does it help the less educated to flat out ignore exclusion terms that are _explicitly_ provided?

Doesn't seem to ignore it for me -- on the first page I only see "headphones" word in the search string itself, in "related searches" and in google maps listing shops that sell them.

Thank you. The rebuttals were good but fail to address this main point. Which was explicitly stated.

Even educated people don't have the ability to hold every possible taxonomic schema in their heads. Don't even get me started on multi-lingual, cross-culture, jargon, creole, and dialect differences.

A cool example I read recently noted that the English word "son" has two different meanings in Spanish: son, a conjugation of the verb ser, to be, and son, relating to sound and music.

Edit: as an aside, I just did a search for son and the first page was almost entirely about the recent horror movie, with one hit for the Korean footballer Heung-min Son

> They needed to help the less educated, too.

Like squeaky-clean trying to read up on a paradox?

Google could have an advanced mode for search or something. Instead they put other information aggregators out off business but did nothing to replace their value. We were better off with the paper version of Yellow Pages for some queries -- at least it was moderated and effort was put into compiling it.

I don't think it is a good business move for Google in the long turn to be a search engine for Wikipedia and Stack overflow and for people that are to lazy to type to top domain for the URL. Forwarding non URLs to Google search in Chrome might be the smartest dark pattern from Google ever.

Regardless of actor causing the issue, Google is the party that serves the search results and thus are responsible for what they display.

Right. Google is the stage the actor gets to perform. Google built the stage. It's like the phrase "if you build, they will come" came to fruition.

The thing to remember as a Dev, no matter how well intended your project is, users will always find a way to use it in a manner you never thought about that is the most advantageous way for the user.

Isn't the fuzzying of the query and expansion of the results, including prioritizing "natural language" querying a deliberate UX choice ? I don't get what the marketers have to do with that part ?

Is it? SEO spammers seem like they'd have an easier time making a page with every permutation of the search string compared to legitimate authors who would likely have only a single variation

IMHO it’s both. Obviously people spend a lot of effort reverse engineering Google’s ranking algorithms because it’s profitable but some of it, at least from the outside, seems self-inflicted.

Image search has been useless for me for a long time because of Pinterest. Many times Pinterest will show you an image but when you click on it it takes you to a page with completely different content. Maybe Google has a good reason for not penalizing Pinterest, but it’s weird to me that they are not more aggressive in sanitizing the quality of their searches.

I understand that google is constantly at war with SEO-people who are motivated to abuse the hell out of search, but why can't they un-fuck verbatim search and give me back the ability to completely remove particular domains from search results?

I'm somewhat sympathetic to this but Google is an enormously profitable company which also is a world-leader in software development and ML, which you would not know from looking at search results.

It also seems relevant that Google has actively removed features like the ability to report SEO spam or block domains — if they cared about the user experience, you'd think there'd be an easy menu with advanced 1990s technology like “Never show results from this domain” or “Report this result as spam”. Yes, there are lots of SEO spambots out there but there are also millions of people who would zealously report spam sites for the many queries which are basically useless now.

As a simple example, on the weekend I was searching for manuals for my discontinued refrigerator. There are a ton of bottom-feeder sites which will list any part number imaginable and some which are even using fake forum sites with templated exchanges between users where someone gives a link to a malware/phishing/porn page followed by several other fake users thanking them. Google has chosen to make reporting things like that almost impossible, so unsurprisingly they remain common. I suspect that's due to their desire to avoid paying humans to distinguish between valid complaints and fake complaints from competing SEOs but either way from my perspective the results are unambiguously worse than they were a decade ago and I have no reason to believe anyone will care about this unless revenue slips.

I wish there were a way to tell Google that I never, EVER want to see an image result from pinterest.

You may not be able to tell Google directly, but uBlacklist works reasonably well and has regular expression support so you can block all TLDs. I added

/pinterest\..*/

and here's the (lack of) results when I google image search "pinterest fish": https://i.imgur.com/pMQ2JQK.png

https://addons.mozilla.org/en-US/firefox/addon/ublacklist/

As others mentioned, definitely both.

One curious example for a non-commercial term the other day was a search for 'population nairn'. It returned a "people also ask of":

"What is the fastest town in Scotland"?

Answer: walkwithdinosaurs

Nonsensical question and answer.

I get this from their perspective. Their goal isn't to work just for people whose work makes them extremely precise in phrasing. I am sure that a lot of the mechanisms that make me bonkers are ones that make search results better for the average user.

But it's still frustrating when my skill in using words to trick machines into doing what I want gradually becomes useless.

it's so bad these days. If you are on mobile good luck finding anything of use if it doesn't involve ads somehow....non affiliated content, useful personal blogs - high quality ad free pages....good luck!

it's to the point where I find what I want when I search torrent search engines.....and not via any search engines on the web.....

90% of my searches for stuff like "best bike to buy under $500" go something like: "best bike to buy for under $500 site:reddit.com"

I do similar things for "real" reviews.

But it's just a matter of time before the astroturfers start marketing more effectively via Reddit for SEO purposes.

That's "getting a stupid answer" because of the kind of question. The problem is you can't rank bicycles under $500; they are all undifferentiated. The search results on Google for this question strike me as fairly reasonable, given the impossibility of the question and the deluge of affiliate spam they have to index.

> that their algorithms assumed you wanted

What? That's the whole purpose of search algorithms isn't it? I'm not sure I understand your complaint.

not necessarily. the assumption part is the key. If I search wild turkey, I would accept listings on the bird or the specific whiskey. I would not accept listings of other brands of whiskey or scotch or other alcohol. I also would not accept listings on birds unless there was direct mention of wild turkey in it. Assuming I was interested in something other than what was requested is where it lost usefulness to me. Maybe others found it more useful, but when I search for a term I want that term in the results when I hit search. It's an entirely different concept when I hit Feeling Lucky. At that point, I'll accept any of the previous examples

The Google search results for wild turkey seem extremely reasonable to me. Did you try this example before using it?

it's just an example. of course somebody on the internet did the exact thing i was meaning to be an example. kudos

the point of the story is that I'll take "anything related in the slightest to my query for $500" when hitting the Feeling Lucky button. when I hit search, I want specifics to my search term. don't go making 6 degrees of separation results.

If you google "how do i search the web without e commerce results", you get "how to improve search in your ecommerce site".

If you search "examples of bougainvillea not pruned", you get pruning tips.

Often the first page of results will already have the tag "Without: --keyword that changes everything but is inconvenient for the algorithm--. Force include this word?"

We're going to end up going back to community sites. Reddit is obviously the good example here, but you can guarantee if you find a 'racing bike' forum (or whatever), take the time to join up, search, ask - someone with a passion for it will give you a great, well researched list because they love it and take pride from knowing.

Exactly the same as when I ask my friend who's passionate about wine for his recommendation for a £15 bottle of red, and he'll happily waste 20 minutes of his day writing a list and sending me links. I know the pleasure he gets when I message and go "thank you! This is amazing!"

Also just to add - this is exactly why those "get paid to recommend things to your friends" startups never work. Because the minute we know that the 'friend' on Facebook who's shouting about the great bottle of wine is getting paid £1 per sale we deliberately don't buy it. (As opposed to the "next time you're in Tesco, pick up a bottle of XYZ, it's great" message - when we probably do.)

I want you to be right but the GME and AMC nonsense doesn’t instill a lot of confidence that Reddit isn’t being gamed already by automated accounts. I’ve had my own comment altered a little and regurgitated by a bot before, which was extremely creepy.

But you are right that the signal to noise is at least better on Reddit.

I do think it is slowly deteriorating though. I've noticed it on indie gaming subreddits. A dev will announce a new game and I suspect a lot of the comments are fake. Maybe not bots just yet, probably more likely the dev gathered a large group of friends to help out. But it wouldn't surprise me if this type of promotion became more and more bot driven as time goes on.

I don’t know. It depends on the ‘quality’ of the community. Eg when I’m considering a new IT tool, I tend to search ‘!hn $tool’ just to see what the comments on HN are like, as it’s the most reasonable signal I can find vs google. Similar example, when searching for a good EV charging pass my first go-to were the Tweakers.net forums. It’s a place where the general IT crowd of the Netherlands hangs out and indeed someone already wrote a very extensive topic on the EV charging world, the business models, what to watch out for, etc.

I suppose if you have the same for cooking, cycling, cars, etc. It might actually end up going that direction for more people.

As a community becomes higher quality though, it also becomes more interesting for scammers to abuse. (The Bitcoin Reddits you mentioned are a great example). Good moderation is key I think.

The bots can definitely be frustrating.

I like to stick to smaller subreddits as they tend to be more authentic. Usually anything under 100K subscribers (with exceptions) can be really enjoyable to lurk + a great source of info.

The GME/AMC nonsense will get downvoted on more serious subs.

The one that started it is called “wall street bets” and people openly call each other “retards” and “autists” over there.

The GME nonsense was a guy posting that he bought a lot of options and why.

Then he kept posting updates

Sure but then it kept going with millions of posts with diamond hand emojis that had me deeply in the uncanny valley.

I bet Reddit could take a big chunk of Stack Overflow's market share.

Reddit would need to fix their UI so that it isn't actively hostile to people trying to read content on their site.

StackExchange is a breath of fresh air compared to the rest of the web. I saw a meta post the other day where they were soliciting feedback about how make their mobile site more information dense. Love it!

Yes! I hate the move to information sparsity and hiding functionality to make things look “cleaner”

Shame most of the content is moderated to an epic level of uselessness.

old.reddit.com exists, for now. Reddit as a company is actively hostile to users, so they don't consider the new UI broken.

Half of the time when I click links on old.reddit.com, it just redirects me back to the home page. When I go to those same links, on the new site, it takes me to some image gallery page. They've definitely broke a good portion of the old site.

people sometimes bemoan stack overflow's rigor and pedantry, but try to ask a technical question on reddit... unless you're in a very well moderated reddit you're getting worse answers or no answers at all

yes! probably only a matter of time until they ramp up the astroturfing (which already happens a fair amount). It seems pretty common for someone to do a Google search for "X product review reddit"

Going to? I've been including "reddit" in the majority of my google searches for years now.

It’s even in the search box autocomplete frequently.

> We're going to end up going back to community sites.

For me they never left... It makes lots of sense: many forums have a "no commercial posting" policy and the users/mods there are very good at detecting submarine/hidden commercial content. To me forums are a better web than the web.

Many of the forums I used to visit in the 2000s are looong gone. I have a lot of respect for the ones that stuck around. For example I am have been on a soccer team's messageboard for over a decade, and seen it go from hundreds of daily posts to maybe a few dozen now.

I like how the show "Halt & Catch Fire" used this topic as a plot within the show. You had the Google-esque team of laborious devs unhappily stressed on coming up with the perfect algo, then you had the Yahoo-esque team of party atmosphere humans scanning the web for things to highlight/promote/scan/etc.

> if you find a 'racing bike' forum (or whatever), take the time to join up, search, ask - someone with a passion for it will give you a great, well researched list because they love it and take pride from knowing.

Agree 100%. And in the "good old days" of the internet (that is, after Google had just replaced the pre-PageRank search engines (Lycos, Altavista, Yahoo)), what you'd find with a search would be precisely those community sites run by fans and nerds that were happy to disseminate their arcane knowledge.

Today, you have to wade through pages and pages of SEO and Amazon-affiliate crap that has no expertise and interest in the underlying subject matter at all.

So here's a side-question. The original key idea behind Google's approach to search was page rank : how many, and what kinds, of other sites linked to this one (and potentially, in what context).

Does anyone think that this can possibly be at the core of Google's page rank anymore? Most pages don't have any significant number of links any more, and if they do, they are presumably garbage most of the time. Who is out there writing articles called "How to find the best bicycle for under $500" and providing links to excellent, current reviews of bicycles under $500? The answer might not be "nobody", but my instinct is that it's so small that it's effectively the same thing.

Consequently, at some point, Google switched from trying to leverage human judgement (i.e. how many other people link to a page), and have been forced (?) to replace that with their own assessment of a page based largely on its own content, metadata and outbound links.

Whereas in the 2000's, you could blame people for crappy results from a Google search ("why do people keep linking to that stupid article"), at this point it seems to me that everything, absolutely everything about Google results can be laid at Google's feet.

I agree with much of this.

I believe the antidote to this is simply acknowledging small communities curate the best content, so essentially marry many small decentralized communities on a searchable platform. Anonymous but karma/identity weighted profiles, curating and debating lists of their favorite things around a single topic.

Aggregate them into a single master index with nice UI, then pay out equitably based on performance.

There’s a lot of questions on preventing spam and fraud, and I think the further away you get from “high ticket products” the more you avoid that problem, but it can’t be worse than Google: in one you’re relying solely on magical AI and links, the other you can have as much magical AI and links as you want, but you also have user accounts which maybe invited each other and can vouch for each other at a cost, votes that can be weighed by all sorts of factors, and basically a much richer graph of self-moderating people to back the index.

Working on something like this at the moment!

>... simply acknowledging small communities curate the best content

This is true for some things. But for reliability reviews, for example, there's a certain power in large numbers. I'd much rather see reviews for common appliances from 200k people (or more) than the same from a group of 2000 appliance-centric folk. That's true even though the 2k set will likely know and care a lot more than many of the people in the 200k set.

On the other hand, with large numbers of people, it's much harder to deal with "95% of reviews are complaints" problem, which tends not to be so much of a problem with "enthusiasts".

Why not both? Rotten Tomatoes gives the critic score side by side with the audience score, as does Metacritic.

There's arguably value not just in both scores, but also both being presented side by side.

Rotten Tomatoes is the epitome of why trusting expert ratings on their own is a bad idea. The number of times that the tomatometer is wildly off the general perception is insane, enough to make the tomatometer largely useless for myself at lease.

It certainly does seem, though, that a company with Google's prowess in search, NLP, ML, algorithms, categorisation and the like should be able to do a lot better, even if the problem is incredibly difficult.

My only explanation is that it doesn't suit the business needs of Google to do so.

Part of the spam is clearly condoned by Google, probably for a mix of reasons.

An example is Pinterest, Google can drop or deprioritize their properties, like they do with other actors. But for some reason they do not. Maybe they are scared of litigation.

> acknowledging small communities curate the best content

Agreed. The problem is that Facebook and Google more or less successfully disintermediated those small communities of old, namely newspapers, magazines, websites based on region or interest which had a vested interest in their constituency. In doing so, the tech giants swallowed up the advertisement dollars going to those old media, and made a few people insanely rich while destroying the entire ecosystem.

> Does anyone think that this can possibly be at the core of Google's page rank anymore?

Yes, it is, and it still makes up the super majority of factors. It's the reason why people buy expired domains (that have links pointing to them), it's the reason why they pay lots of money to rent a subdomain or folder on cnn.com. It's the links, and it's only the links. And it works great.

The actual content on the site has little to do with the ranking. Yeah, sure, you need to have the keywords in there, but you don't need quality content, you just need quality links.

So you're saying that the page rank for http://foo.bar.com/baz.html is more a function of the link score for foorbar.com than the link score for baz.html or its contents?

No, but you do have internal links on your site as well, and they work just like external links, passing on "link juice". If you have a million important links pointing to / and zero external links pointing to /baz.html and you link from / to /baz.html, you'll still see /baz.html hike up the SERPs.

https://news.ycombinator.com/item?id=27991322 this submission from today is exactly the same. Expired domains are the big business these days, and they can absolutely catapult you into the top 3 without any change in content, performance or layout.

This seems quite contrary to Brin & Page's original explanation of their (first) page rank algorithms, which was boosted as utilizing other people's linkage as the justification for a higher ranking. It also seems ripe (ruined, even!) for being completely weaponized. Another failure of the imagination, maybe.

It makes sense though, I believe. If you link to the home page of a website, what would that mean if you consider it a vote: "I endorse this precise document", or "I endorse what's on that website"? It would make sense to lessen the endorsement if you link to individual pages only, but not the home page / hub pages.

SEOs have, of course, gone to extreme lengths with that by doing "pagerank sculpting", i.e. making sure you mostly link to the pages you want to rank well with, and not your privacy policy. From my experience, they've stopped being overly sensitive about that now ("it has to look natural"), and it's just about raw power: who can buy the best expired domain (state entities being dumb and just letting their domains expired when they rename...), or who can rent a folder (and the links that come with it) on the strongest media platform.

It's a cat-and-mouse game, and google seems to not want to be a cat unless they can be a perfectly automated cat, and they can't.

I see reviews of best X under $Y that I would have some trust in their conclusions on websites that also do news about the subject area. I don't see why these websites should not be able to use affiliate links.

Some examples: Cycling Weekly for bicycles and AnandTech for computing stuff.

I don't go to their websites via Google search though.

My impression back when Google really took off was that other search engines were basing their relevance scores on page content and getting lots of spam in their results from pages that gamed the relevance algorithm. Then here comes Google using human judgment via PageRank and getting great results, because who would link to garbage?

Of course the web has changed since then. And I think you're right on in thinking that the signal-to-noise ratio of links has gone way down. But it may not be fair to chastise Google for not getting good results just from page content. Has anyone ever done a good job of information retrieval in an adversarial environment without leaning on human judgment?

> I think the answer is because it’s not in their best interest to do so. They like that most consumers will look at these reviews, click through to Amazon and make their purchase with little hesitation.

I don't understand the logic here. Why exactly would Google want people to thoughlessly click on Amazon affiliate links from other sites? There's no short-term beneft, since they don't get a cut. And there's no long-term benefit, since it just risks teaching people to go straight to Amazon for commercial searches.

I think it’s less in Google’s long term best interest and more a marriage of convenience. Hopefully a temporary one.

Google increasingly wants users to find what they want on Google itself and not have to click through to the direct search results. Featured snippets are the strategy and these affiliate site listicles are crafted explicitly to be hoovered up as such by the algorithm.

One hope is that long term Google improves quality-content detection and deprioritizes the garbage.

> There's no short-term beneft, since they don't get a cut.

But they do? IMHO some of the SEO techniques use direct payments to Google for keywords, results placement, etc.

You can buy an ad, but those are shown differently than actual results. You can't pay Google to improve your SEO.

I stand corrected, thanks

But those ads can increase the number & frequency of pages crawled by Google.

agree on that front -

but giving mediocre vs high quality results on the other hand forces you to use a particular search result page more and increases the likelyhood that you'll click through a paid search link in your frustration ...

But again, the author's theory is that what's being optimized for is to get people to mindlessly click through an Amazon affiliate link and making a purchase. If people are doing that, they clearly aren't doing more searches like you suggest.

"The search results are intentionally bad to make people search more and click on ads" would be a viable conspiracy theory in isolation, but seems to directly conflict with the complaint of the search results driving people to make purchases via the pages that showed up in the organic results.

    And there's no long-term benefit, since it 
    just risks teaching people to go straight to 
    Amazon for commercial searches.

Yeah, I came here hoping somebody would have an answer to this seeming non-sequitur.

Google has been utterly useless for "what is the best X?" product search stuff for years, for exactly this reason.

The result in my case is that I've learned simply to not use Google for these searches. That is not good for Google.

The author says "Google, like many of the other tech giants, benefits from an uninformed user base" but I fail to see how that is the case. They benefit from a happy userbase that finds Google useful and therefore trusts Google with their time and personal information.

Sure, Clueless Cletus or Simple Sally might buy a bike from that clickbait bike article but I'm not sure they're walking away from the experience feeling that Google was particularly useful or helpful and wanting to rely upon them in the future.

> Why exactly would Google want people to thoughlessly click on Amazon affiliate links

I don’t think it’s specifically Amazon links, but if they’re mindlessly measuring metrics without thinking about “why” things happen, the affiliate link spam that dominates results might increase ad buys. Think about it. If you can’t rank on the first couple of pages due to affiliate marketing spammers, you might buy an ad. If you’re competing with some of those spammers for ad buys, you might pay more.

It’s just super poor leadership and a lack of critical thinking that has let it happen IMO.

So we've an "if", a "might", a "might", and one more "might" with no supporting evidence, and that's directly contradicted by the screenshots in the article. Specifically, if the goal here was to bury the good content so that people would be forced to buy ads for it *there should be some ads pointing to said good content*. There aren't in the screenshot.

I would suggest that you didn't really think this through. You've decided who the villain is, spent 10 seconds coming up with a illogical narrative where that is true, and then just state it as some obvious fact. Just like the author of the original article.

The first hit he gets is a website plastered with ads. Boosting sales of ads is still a plausibele explanation.

"But I bet it has something to do with the fact that all of the bikes they review are sold on Amazon and that every link uses an Amazon affiliate tag. It goes quite well with the dozen or so ads plastered across the site (some of which are autoplaying videos with sound)."

I believe both search and affiliate programs have flaws... but I didn't see anything that convinced me this is some conspiracy between the companies.

I also don't have a problem with affiliate programs. If someone posts links that are relevant to the topic at hand, there is little harm in them getting a few cents from a purchase I would have made anyway. If anything, it is funding content creation in a way that works.

That being said, the point of the article is 100% correct that low-effort, low-value review sites just to get affiliate links ranked in search results are problematic and annoying. I make a judgement call when I run into them - if they really did put in some effort and have decent reviews contrasting different items, I'll go ahead and click the link. They wrote content, the content was helpful, no problem. On the other hand, if it was just a list of links with no added value, I don't click the links. It is just as a easy to do my own search on a product name, or even copy the link and remove the affiliate tag.

Like everything else in the world, nothing is black and white. The presence of some problematic sites doesn't invalidate an entire mechanism for funding content creation.

> If anything, it is funding content creation in a way that works.

Affiliate programs distort content, to the point of being useless, or justify making of outright fake content.

I would argue that google is being used incorrectly here. Google is specially being asked for a product list which meets a certain criteria. The user is asking Google to recommend the best bikes..

The user is actually looking for authoritative bicycle reviews to make a purchasing decision. Searching for “bicycle reviews” would give a far higher chance of finding what the user is after. They can then use the criteria on the review site to narrow down what they are after.

There is also a limit to what searching will get you. Going to a bike shop will allow an expert in this subject matter to assess your personal requirements and advise accordingly. For a lot of things, going out and talking to someone will yield far better results than Google ever will.

While I agree with you about the incorrect search being used, I disagree that bike review results will give anything more useful. Many "reviews" are also collections of affiliate links, disguised as a review.

> Searching for “bicycle reviews” would give a far higher chance of finding what the user is after.

Nope. Google will still give you dumb reviews pages full of “Check price on Amazon” affiliate links. What works for me is adding “site:reddit.com” or “forum” to search.

Some communities have their own forums they use too, and I’ve only stumbled upon these by accident. It would be nice if there were an index somewhere that kept track of them.

did you actually try that query? I just got lots of relatively high quality review sites/periodicals. I say "relatively" because even industry stalwarts like Bicycling Magazine, which has been around in print and/or online form for decades, are ad-driven and far from unbiased. But if you trust Consumer Reports or NY Times more, they're on the first page too.

just tried. Got surprisingly sane results :) But what I said I experienced over last year for practically anything else than bikes, all electronics stuff was item one, item two, no real opinions or substance, and invariably "check price on Amazon" buttons.

Blame it on the user is always winning strategy, not? Google prides itself on its semantic search that can peak in the depths of your soul and understand your most hidden desires. I'd disagree and will say that asking for best bikes should give you a list of bikes that many people agree that they are good.

I believe that I will create a new internet law: All "reviews" pages will eventually become a list of affiliate links.

For more information see this article on mattress reviews [0].

[0] https://www.fastcompany.com/3065928/sleepopolis-casper-blogg...

There's also an even worse thing here. If you search for a product under shopping on google, something that is somewhat known like a certain TV model, and you'll see most SHOPPING results are from websites that actually don't sell anything, but rather will have a "product page" where either the buy button is disabled or redirects to amazon. They typically have much lower prices, or just low enough so you'll click.

From a certain perspective, you can argue Amazon has a monopoly on sales coming from search engines.

I don’t understand why Google doesn’t make Google shopping not terrible. Big potential business value for them if people actually used it.

Hm, this is not really news, but perhaps it's the first time I consciously think about it.

The article is right. I used to search Google for what the best X is. At some point. Indeed, I also used to search amazon and read the reviews.

Now, and for what feels like a while, I am not doing that anymore. Like, at all. And this is not a decision I consciously made.

If I want to know what the best dishwasher is, I am not going to amazon. I am not putting it into Google, simply because it's all affiliate links all the way down. I either read professional reviews (like non-profit and established test companies, like Warentest here in Germany) or I ask the community somewhere.

Google and Amazon have conditioned me to subconsciously not use them anymore for that purpose.

Very interesting.

Buried in there is also the fact that getting good and reliable information might indeed not be cheap (could be paid for by the public/donors if it is a non-profit, but still costs).

In the earlier days of the web and search that was not so much the case, I reckon. For example, a lot more paywalls than 10 years ago.

I use Reddit a lot for this.

But this is an arms race with SEOs and bots. Eventually Reddit will probably be worthless for this.

What makes you think what you're reading on reddit is not already socially engineered advertisements and bot spam?

People have been paying attractive people to "hang out" in bars and pitch gizmos and doo-dads for decades (citation needed); absolutely there are accounts on reddit shilling gizmos across relevant subs.

Oh sure, but there's still some use in it. Eventually, it'll be all bots and no use.

I still use Google, but with "reddit" or "wirecutter" or whatever else might have the information I need tacked on the end.

Same here. When I need information on a product I want to buy, I will either go to a specific subreddit or, in the case of tech, a YouTuber that I trust to see if they have a review of it.

Amazon and Google are certainly not where I go to find credible information on products anymore. I advise my friends to do the same now

Case in point, I need to buy a new medium sized kitchen knife. I essentially know what I want, but I went to Amazon today to check out the state of things.

I am in Germany, however, on the first two (!) pages of search for "kitchen knife", only two companies that I actually know make knives are present (the big ones: Wüsthoff and Victorinox).

ALL other results, that is over 80 results, are fake Chinese companies with "German like" names, essentially selling the SAME product - a fake Damascus (laser etched) knife with wooden handle. Most of them have 53+/- something HRC (probably widely fluctuating around 50 HRC) and are made of "German (style) steel", whatever that means. Indeed, names and presentation suggests that the knives are made in Germany. They are not. Sometimes, the company is registered in the EU, but if someone uses the question function to ask where the knives are produced, it is always China. All of them have fake reviews, naturally.

I mean... how useless Amazon has become to order even a midrange price good, like a decent but not top range kitchen knife. Now, I am not in principle against buying goods from China, but I can use Aliexpress myself thank you. We also all know the quality of these products when they arrive, and the inability to return them (as the price point is chosen that this is not worth it).

What I hate about it is the constant deception. Nobody would buy a knife if it would say "Price 10€, made in China". So instead, it's 30€, the name is something like "Schneidwerk" and the Amazon page is littered with German flags (ugh), whereas the company probably does the same in every other European country (with, for example, French or Italian flags).

Somehow, this makes me really irate.

It gets even worse / weirder - I've found if you search for products that might not even exist, you get made up lists of products.

A while ago, I wondered if there were such a thing as a microwave + toaster oven combo (there are several convection oven + microwave combos on the market, but none can toast like a toaster oven).

Here's the top non-sponsered result: https://www.chefspencil.com/10-best-microwave-toaster-oven-c...

And the second result: https://thebrilliantkitchen.com/best-microwave-toaster-oven-...

Sure sounds like there's at least 10 of these items in existence for me to choose from - but none of the items in these lists are actually both microwaves and toaster ovens.

I've even had this with mis-spelled words (i.e. "the best Bivycle of 2021" or something like that); I wonder how the f*ck those sites are being generated.

> I wound up buying a used bike from a local shop

I think this is a positive outcome. Google and Amazon are driving people to shop locally with their tricks.

Same thing happened to me and I couldn't be happier I bought an electric bike from a local shop. I probably paid a little more, but I made a new friend. The local shop owner is a really cool guy who was interested in meeting my needs, not in selling me his most profitable bike.

I add “site:reddit.com” to consumer/product related Google searches.

This usually gets me the quality information I’m looking for.

Or simply add "Reddit" to your search query. The first result will be from Reddit. And there will be a URL to other results from there.

I see more and more queries suggesting autocompletion of "reddit" and similar sites these days. Google's default results are extremely poor quality for many queries.

I guess many people do that, but it shows the review system failed miserably when you look for quality information on reddit. I mean, in theory, reddit would have all kinds of noise and fake opinions, whereas Amazon etc. should have only verified and curated reviews, including methodologies for spotting abuse and procedures for manual verification. Well, the economics of scale that drives them insane profits is also the cause they lost this battle.

Reddit is always going to be better. Niche subs are where actual experts who've tried entire ranges of products hang out, amazon is where one random guy bought their first crappy product and left a 5 star review because they don't actually know whether what they bought is good or not.

Or even worse, leaving a 5 star review because it is what they ordered but they haven't actually used it because it's a present for their nephew...

My pet peeve is nagging to leave a review as soon as possible after the purchase. I mean, what can I write at that point? I received the product. It is blue. The seller shipped the product quickly. That's all.

Personally, I'm exclusively interested in reviews by people who have used the product for at least a couple of months, preferably years. But I guess the review score would be much lower and the impulse to buy would be weaker. And its a shame as there are products that deserve excellent reviews.

More likely leaving a 5 star review because they are working for the manufacturer, or left a 1 star review because they are working for the manufacturer of a similar but different product.

If consumers start relying on niche subs en masse, then it is only a matter of time until Reddit is overrun with shills and sock puppets.

The mods of subs for the most part know what those look like as do the existing communities. Most niche reddit subs are very sensitive to astroturfing because the people involved in it rely on good information from those subs, they also know who the regulars are as well.

And so we move on to other niche forums. It's going to be a very drawn out battle lol...

Google search improved web but then ruined it for no fault of its own [0]. There is just so much spam.

[0] https://en.wikipedia.org/wiki/Goodhart%27s_law

Except Google still displays a lot of archived very old threads as if they are new[0] so often searching for recent reviews that way is annoying.

0. https://svilentodorov.xyz/blog/bad-search/

I do the same, but I think there are an increasing number of astroturf campaigns taking advantage of that, too.

There certainly are, and they are growing more sophisticated.

Still better than print media though.

Are there any nice Chrome extensions to expand on Google that anyone can recommend? Would be nice to have a default Reddit setting, or a 'search reddit' button on Google.

There's a couple other ones as well besides Reddit, particularly around key forums. Shouldn't be too hard to allow custom-buttons.

In general now that people are here, I'd be pretty interested to know what people's top 5 extensions are outside of the regular (uBlock, Pocket etc). They're quite powerful but I don't use that many actually.

In the EU I use a no-cookies extension, though imperfect. I also use 'video speed controller' religiously to easily watch/listen to content faster. But overal I find few extensions very useful and the ones I use cause issues on a small fraction of sites. Happy to hear from computer super-users what they use.

> Are there any nice Chrome extensions to expand on Google that anyone can recommend?

Just have to point out the irony of patching google’s spyware browser to get around google’s poor results

Yes but unfortunately it still creates links using the “new” Reddit interface. Installing an add-on like Old Reddit Redirect helps to restore sanity when searching by site.

Affiliate links are the reason anybody even makes a website anymore. The era of personal websites made for intellectual curiosity and expression are long gone.

It's not just websites though, Youtube and IG are filled with them as well. Basically, if you have any audience at all, it's time to cash in on them by polluting your online presence with these dumb links.

Whenever I want to make a purchase online, I search for "best _____" and add reddit as a keyword. Reading through a few threads on reddit will typically give me a pretty good idea of where to start.

If it weren't for Reddit, Stack Exchange, etc. nothing on the Internet would be organic anymore!

Try https://millionshort.com/search?keywords=best%20road%20bikes... If you search Million Short then the top two results are (1) an actual bike store and (2) an actual bicycling magazine that's 100+ years old instead of seo spam.

But you raise a much more important point: if you query the web like a stupid person then you'll get stupid results.

The OP shouldn't have searched for "under 500" because good bikes under 500 are called used road bikes from a local shop or craigslist owner (as the author revealed and I discovered myself when I was younger c. 2006) so saying "under 500" is sort of like saying, "show me the cheap mass market bicycles please", because the number of people who want good bikes is small, but it's a really devoted group, who are willing to pay what people like us would pay for a good PC or even a car, so if you've already decided you want the chromebook of bikes, then amazon and google aren't doing you that much of a disservice, let's be fair.

I think the fundamental problem is that any sufficiently influential system which provides consumer information on products or services becomes an obvious target for those sellers and service providers to try and game.

I think there are basically only 2 ways around this

1) a consumer reports type model were consumers must trust a central authority to be inviolable.

2) legislation that harshly punishes companies who try to influence their own reviews. These would likely be ineffective unless they were incredibly stringent.

Is there a more reasonable model that can avoid being gamed though? I feel like more and more I have to rely on word of mouth about products which isn't great for different reasons.

I think a bike is one of those items that you have to try out before you buy it, so I would not buy it online and I also would not go to Google to research it (at least initially).

When I was in the market for a new bike a few years ago, I looked at Yelp to find the 5-6 bicycle shops with the best rating in my city. Then I went to each shop and told them what I wanted from my bike and my budget.

This process went very well. Not only am I supremely satisfied with my purchase, even after a couple of years, but I also found a new great bicycle shop that I go to for repairs and upgrades and stuff.

I agree with try-before-you-buy for bicycles - size, style, etc make a massive difference in utility and comfort.

But, how would somebody who only had a bike as a child know this? Google isn't telling them - it sends them to Amazon via listicles. If they happen to search "how to buy a bicycle" there are some potentially useful results. But, even if they do end up a bicycle shop, their likely to run into inventory problems - many bike shops simply don't have any bikes right now. So, it's back to Amazon, or Craigslist/FB Marketplace, or worse.

I think search is broken full stop for any search in the form "Best X in/for/under Y" - there are too many low quality sites publishing 'list' articles, which just link to items for sale that offer them affiliate commission.

For the example of "bikes under $500" it's often better to do a google news search to get magazine/newspaper articles where they are more likely to have done actual reviews.

Totally agree, I've been impressed with how effective adding "-intitle:best" to a Google search is at filtering out SEO-gaming fluff

A huge number of developers at Amazon and Google are clapping their hands with glee when the read this…. It’s their jobs to get exactly this outcome. They’ll show your post to the boss to lock in their bonus.

This is the prime purpose of these gigantic companies. Of course it works like that.

The smartest software developers in the world dedicate their working lives to advertising.

I don't get why you're downvoted. That's exactly why these things are failing.

This sounds so familiar. I used to scour the internet for reviews before buying anything. Analysis paralysis is real, and I sometimes just give up buying at all.

Over the past years, I found more bad review websites than helpful ones. I doubt there are any real unbiased reviews.

Go to google, enter the ad trap Go to your local store, you have to deal with the sales target of the store. Go to Reddit or a forum, and the people there just want to justify their purchase.

It is all fine as long as you know it.

The authors solution to just go to a local bike shop and buy a bike is in my opinion still a reliable best. I assume he got the opportunity to test ride the bike before purchasing it.

My new approach is to write down what I want from the product. This gives me clarity. I still read reviews but take them less serious.

I either order online and once I receive my package I sent it back or keep it. Alternatively I just go to a physical store and try it out before buying it.

The reason affiliates always tend to use amazon's program is not because it has the highest pay outs. Instead its because Amazon converts really well.

I've been noticing this too. I don't even remember what I was researching, but I was looking for "best of" lists like OP and I kept finding affiliate-link farms in all sorts of weird places. One was a recipe site, I think, and I definitely wasn't looking for anything even remotely food related at the time.

The root of my problem IMO is that Google never came up with a proper way to search by time. Same problem if you want to search for "new video games" or new anything really. You can't specify it as a time constraint. You have to put the year in as a regular search term, which is super easy for the link farmers to game.

As for the search in question: instead of "best commuter bikes for under $500", do yourself a favor and buy a beater bike for $100-200. The comfort is the same if not higher, but you stop thinking about your bike at all, and if someone steals it you buy another one next day without shedding a tear. My favorite ones are by fahrradmanufaktur, very solid, a new one will cost you a few thousand bucks, but after 5-7 years it still feels fantastic and you can buy it for a fraction of the price.

I really was clueless at the time and figured it would take at least $400-500 to get a decent starter bike. Thankfully I wound up coming across a used bicycle sale in Brooklyn shortly after and picked up an old Giant for $150. Have probably done a couple of thousand miles on it since!

We have a small used bike shop nearby, damn great value for money! Bought a used Merida MTB for my so and a Bergamot each for my daughter and myself. Condition is close to new, cost in total 800 bucks, some smaller repairs included.

With the bike shortage the last two seasons, used bikes in OK condition can be almost more expensive than a new bike (since the new, unused bike is sold out everywhere).

At least where I live, there is no chance to get what I would call a usable/durable bike for "$100-200" at the moment.

I bought a couple of them and parked them in different areas of the city to make it easier to reach any location using the subway + bike combination. I choose the ones with flaws that would deter normal buyers but relatively easy to repair, or with a damaged part that I'd replace anyway. I asked a local repair shop if it's fine if I ask the seller to send a bike directly to them and they happily agreed. So I look for an occasion, buy it, they ship it to the repair shop, the shop guy calls me and we talk about what to repair/replace, and in a few days I have a nice bike that will last many years with proper maintenance without spending a fortune.

I reject the author's conclusion. Either this is a non sequiter, or I'm missing a major piece of the puzzle.

    How is it that a search giant like Google, with thousands 
    of brilliant minds, can’t help me find good products to 
    buy? I think the answer is because it’s not in their best 
    interest to do so. They like that most consumers will 
    look at these reviews, click through to Amazon and make 
    their purchase with little hesitation.

Sorry, but what am I missing here? How does Google benefit from a clueless user thinking this article is reliable, clicking on an affiliate link, and buying a bike at Amazon?

As I understand it, Google derives value from their search engine in the following ways:

1. Directly, via paid ads and product placement.

2. Indirectly, by simply being useful, which means people will enjoy using Google search results, which means they will use them again in the future, which is what allows them to sell ads and gather data on users.

So as far as I can see it's definitely in Google's interest to provide good and useful search results.

> So as far as I can see it's definitely in Google's interest to provide good and useful search results.

A shame Google didn't get this memo, or buried it somewhere.

It's so fundamental to their business though, right?

They are fundamentally a data mining and advertising operation and that only works if people find them useful and therefore use them so that they can be data-mined and advertised to.

I guess maybe Google just feels it's an unsolvable problem, or has a backdoor agreement with Amazon, or feels they are so synonymous with search that only a fraction of users would ever user another search engine even if the quality of their results has become awful

How many hairs do you need to remove from a man's head before you can call him bald?

Many chain restaurants follow this scheme: when they appear and need to make a good name for themselves, they are hell bent on the quality of food and the the customer experience. It's important for them to get the word of mouth marketing. But after they establish themselves firmly and start to grow, the bottom line becomes a priority, and they start cutting corners ever so slightly. The portions become smaller but the prices hike up. They start skimping on sauces. One day you discover that the garlic sauce is no longer made in house, but is of the same variety that is purchased in bulk for hot dogs at gas stations.

But the momentum is still going.

And many other chains in the same price range either don't make the same kind of food, or are worse still. So you have no choice until a new restaurant starts that needs to carve a niche and beat it's competition.

It's almost as if it's a really hard problem to solve.

Remember taking pride in being able to find anything with google? Now it feels like "as seen on TV" in the search results.

Step 1: Go to a local store Step 2: ?? Step 3: Throw your computer in the trash

I recently did this with some badminton equipment I wanted to buy. Incentives online are terrible. I went to the pro-shop in my club, talked to a coach who's seen me play and he gave me a few options one of which I ended up buying from them.

Listicles have gotten on my nerves, I googled 'how to remove listicles from search results' and got, yes, 2 listicles in the SERP. I tinkered with ddgr[1] and jq for removing listicles with some success, if someone wants to pick up where I left off:

[1] https://github.com/jarun/ddgr

I do not know if this is relevant to this discussion, but amazon search results in amazon.com are really bad. And they are just supposed to give me product details from the key words I give.

I have to do a google search to get the relevant search result for an amazon product page.

This feels like a misunderstanding of what it means to have a good bike. If you aren't a regular rider, odds are high that no review will actually be relevant.

From how you want to be seated to how hard you can take a climb, bikes are stupid personal. Even tire choice is one that will float above entry level concerns.

Ignoring that bikes are heavy in ebike mode right now. And reviews are, by definition, expensive. I can tell you that I like my bike. I can't compare it to more than a couple of others, because I can't afford that many bikes.

That spam level on search engines is increasing year on year, is a fact. It’s also a fact the Google keeps rolling out several updates but the overall quality is getting worse no matter what.

I ran into a similar thing last year when researching replacement LED spots for my ceiling lights. Most review sites used a similiar headline like "best LED spots in 2020" and they were all comparison tables featuring only products available on Amazon, most of them being manufactured by brands that noone probably ever heard of and that are only available on Amazon, and it was all the same basic crap. Not sure how but eventually I found much a better brand with higher-quality LEDs.

Does anyone have any tips/tricks on how to search using google to surface more relevant results besides appending site:reddit.com?

I also remember the past magically search days fondly..

I'm starting to avoid Google search alltogether but my best trick is including a really specific term that SEO would avoid because it might not be familiar to the reader. 90% of the time it will ignore the term but somewhere on the page there will be a page where it says it didnt match that term and if i want to search for matches that do include that keyword. That sometimes improves the results. Note: the query will look the same as quoting the term but in my experience doing that initially is simply ignores. All the old dorking/google-fu operators have been ignored by Google for some time now.

Slightly off-topic, but I have similar gripes with the Fandom wiki. I am frequently on the Fallout wiki there, but for most of the searches I run (e.g. for in-game characters or items) the first result is a link to a sponsor site. I keep seeing Amazon and Ebay among others. I can't adblock that as far as I know. (EDIT: Switched to uBlock Origin and this method does work on Chrome. Very happy now.)

> I can't adblock that as far as I know.

Yes you can. uBlock's element picker [1] allows you to hide arbitrary portions of a webpage, and I assume (?) other ad blockers also allow you to insert custom rules.

[1] https://github.com/gorhill/uBlock/wiki/Element-picker

I use to make and search for pages like: "List of foo with bar" trying to make the list as complete as I could. A bit like a single page web directory. Google would then complain that I had to many links on the pages and de-list them.

When seriously researching anything at all you want the full picture of things out there. You cant work with random joe's top 10 affiliate links.

There is little incentive for anyone to curate a list of 'best bikes under $500' for any reason other than this though?

However. the fact that you found so many amazon affiliates says more about the integrity of other bike retailers. Why would there not be an even spread of affiliate links to merchants other than amazon do you wonder?

Certainty of payment, that's why.

I don't think there is little incentive for anyone to curate a list of best bikes for under $500. There are plenty of incentives for journalism publications like MCN (motorcycle news) to do reviews of the products in the sectors they cover. There are plenty of publications about Cycling, it's a very popular sport.

The problem is that the monetization of the affiliate system creates a massive incentive for people to crowd out the real people who do this. Instead of googling which bike to buy and being met with a number of fairly reputable publications which specialize in this stuff, you're met with Business Insider and other trash which are all just following the incentive of trying to get their affiliate link clicked between the time you start researching and the time you innevitably buy something off Amazon.

I think the MCN example above is actually really good - since no one expects to get an affiliate payment from Amazon from someone buying a motorcycle, the quality of the reviews for "Best motorbike under $5000" are about a million times better than "Best bike for under $500". The good reviews do exist, you just can't find them, because Amazon has completely screwed the click through path from "Google X product -> Purchase on Amazon"

If people are not willing to pay for content and block ads, affiliate and content marketing schemes might be the future of these sites as well.

I think we may have stumbled upon an unfortunate circumstance of how our brains work, and it's being actively exploited (like say how we agree that marketing cigarettes to kids is bad kind of thing?). We could reasonably say that we are willing to pay for content, we're just not attuned to seeing how "free" content is actually detrimental to us and is being monetized behind the scenes.

At the same time, Google/Amazon latched on to it and are taking advantage of it for their own gains. Either way, we're stuck in a downward feedback loop that's only going to make this worse until one of the big players falls over.

The likes of motorcycle and car magazines/review sites are also incentivised by money, it may not be quite so obvious how though?

Let's say I want to do an honest review of bikes under $500, where am I going to get the bikes from in order to do this? Shall I go out and buy them?

This is covered by standard journalists all the time - Linus (from LTT) talks about this all the time. Of course manufacturers give samples to test, but that's table stakes, there is absolutely not an expectation of specific positive coverage.

> There is little incentive for anyone to curate a list of 'best bikes under $500' for any reason other than this though?

“Back in the olden days” people would write pages because they loved the subject area and wanted to share knowledge/show off/start conversations.

So it was very common for people who were just really into bikes to spend some time sharing their thoughts on what the top 10 bikes under $500 were because they wanted to help people buy good bikes. There was no monetization, or at least direct monetization.

Those sites may still be around, but monetization has driven low-quality content higher up in SEO so the bike lover (who probably doesn’t care about SEO) doesn’t get content seen.

>There was no monetization, or at least direct monetization

Maybe they can try affiliate links! I think that was how the "crappy" sites used to do it in the old days. I never really understood the whole seo game and if anyone actually made money or just pretended to make money so they can sell courses

I think some people like sharing knowledge. I keep a blog where I just write things that are interesting to me and maybe to others.

I used to have affiliate links because I was playing around with stuff, but don’t think any have worked in 10+ years.

I have no monetization plan or ever figure I’ll make money off the site. But it costs me nothing to run, so that’s ok.

There are many sites like this, but they are much harder to find because Google isn’t able to differentiate meaningful content from the facade of meaningful content (ie, SEO).

It’s like the degradation in academic papers where groups will reference each other like crazy to game rankings.

Controversial opinion: this is the system working. These spammy affiliate link sites work well enough that Google lists them, people use them, and bicycles get sold. It isn't globally optimal, but it's locally optimal given a culture where online users refuse to pay for content.

Does anyone know any good alternate search keywords when trying to find "Best x under x?" Content marketers already know that people search "Best x" when doing shopping research, so I wonder if there are alternative keywords that have less spammy results.

Oh yeah, search results are now all about selling stuff. I said as much on a similar story: https://news.ycombinator.com/item?id=27999002

I often search with "reddit" at the end of queries like this to get some good opinions. I'm sure a lot of you know this already though, because "[question] reddit" shows up as recommended search all the time.

I would love if sites containing too many affiliate links were de-ranked in google. Google doesn't get a cut of the affiliate revenue, so I don't understand their incentive to let this kind of spam proliferate.

I would pay to have every page with an affiliate link removed from results.

As long as the carousel of ads and products are shown at the top of the screen, it's all good. Google has monopoly mindshare, it doesn't care if the organic results suck. What are users going to do, go to Bing?

I experience the same. Frequently run into "Top 10 reviews" that appear to be written without the author having having possessed the product and then link off to Amazon. Time wasters.

At some point Google is going to let you simply narrow your search to the dozen or so sites you regularly visit. The rest of the web, outside the top 100, is dying.

People need to to become more informed themselves instead of being ready to click through the first line of result.

May be we need more informed people and less SMARTphones.

I could say that affiliate marketing in general... wirecutter - NYT has a whole site and twitter feed for their "deals" with affiliate links

They are pretty open about the business model, and to put a lot more work into the reviews than just generating a list of affiliate links 3ith the data available from Amazon‘s API: https://www.nytimes.com/wirecutter/about/

The Wirecutter has been accused of changing their recommendations based on their ability to monetise product links as an affiliate:

https://www.xdesk.com/wirecutter-standing-desk-review-pay-to...

Wirecutter doesn’t do reviews of “the best x,” they do reviews of “the best x that can be purchased via an online store that offers adequate referral revenue to Wirecutter”

It's also a rough time to be buying a bicycle. I bought 3 bikes (1 new, 2 used) this year and each one was a minor miracle.

I wrote this pre-pandemic, back in January of 2020. I was able to snag a bike at a used bike sale a few weeks after writing it. Congrats on your new bikes!

Didn't this concept ruin the internet a long time ago? Like think about the web before AdSense?

I say it time and time again Google is an ad company not a search company they just happen to be doing search

Precisely. I imagine people in this thread have never opened Google Ads in their life. There has been plenty of improvement when it comes to their search ads tools.

why would anyone take traffic estimates from a site like SimilarWeb as legit?

All those kinds of traffic rankers like Alexa etc have been uncredible for like 20 years. There's too much bot traffic out there now for hits to be filtered reasonably by an external third party

Does anyone know why this website is being blocked by the Indian government? "Your requested URL has been blocked as per the directions received from Department of Telecommunications, Government of India. Please contact administrator for more information."

Neeva is good for such product searches.

It's missing a (2020).

Affiliate links arent the problem per-se. Shitty content is. Google is supposed to penalize sites like this with lousy, thin reviews and promote things like wirecutter and whatnot.

I much prefer affiliate links to the alternative: Unblockable ads and paywalls.

Also, ebay.

First world problem really. Also people look for deals hence in this example a second hand bike is purchased. There is no 'title' with bicycles and anything second hand bought off someone you don't know has to be assumed to be stolen. The market for second hand bikes encourages bike theft.

There are plenty of very good new bikes at almost all price points and the guy in the local bike shop can show you three bikes suited to your size, your use case and your budget.

Docking about online doing your research is all very interesting but the guy in the bike shop will help you just fine without you being an armchair expert.

No point ranting on about Amazon you just have to get away from the screen and get to the bike shop.

Reply on news.ycombinator.com