How Google alters search queries to get at your wallet | Hacker News Reader

How Google alters search queries to get at your wallet

wired.com

207 pointscdme122 comments

I have no idea if they do it to for ad reasons, but Google has been creatively interpreting search queries for a long while now. I think it's one of (maybe the primary) the reasons why Google search quality has fallen so much.

It's not a huge stretch to think that while they're at it, they would take into account advertisers. Again, not saying they are -- only that it wouldn't be that shocking.

We now have direct evidence that the ads people interfere with the product to bolster sales at the expense of the users:

https://news.ycombinator.com/item?id=37675467

It's a near certainty this is happening in all parts of Google.

I don’t know if it still does it, but an enormous pet peeve of mine was how Google would conflate NSTableView and UITableView in search results.

They’re quite distinct!

I can't tell if you're trying to make some subtle point through sarcasm, because I don't know how those (which I understand to be Apple UI components) relate to searches and I'm not sure if that's because I just don't have the context or if them being unrelated is the point.

The point is that they're unrelated, but Google "helpfully" interprets your search to show you things it thinks you want instead of what you asked for.

That is the obvious interpretation, which for some reason completely eluded me. My mind somehow went down a rabbit hole of whole google structured markup, but those are Apple UI terms, so I couldn't see how it mattered unless it was in some iOS google search app, and at that point I was already in a thought prison of my own assumed context, even though this is so obvious in retrospect. Thanks!

You can wrap search terms in quotes for exact results

It is so extraordinarily valuable that folks like Megan Gray have been attending these trials. This story would be so much more compelling if we could also see that slide — but as a closed trial only those with the time and ability to actually go to the courthouse can see it.

On the same day this slide was presented, she also tweeted: https://nitter.net/megangrA/status/1704244948378194289

> I am so livid at what just happened re public access in the Google Search antitrust case right now that I don’t trust myself to tweet about it yet. Back later.

Megan made so many assumptions in that article based on a slide and single quote that we can't even see.

How about they actually provide some evidence before making such claims?

Concrete question: how does the query rewriting service talk to the ads DB to do what's alleged?

We don’t know how many assumptions Megan made or even who was testifying and what they assumed or what evidence was presented in court. Yes, that’s problematic.

It happens it is actually an evidence or key exhibit, in an anti-trust trial.

She could be surely sued by Google by reporting false events.

So I guess we can trust her reporting.

What?

Why should I trust her reporting? She gave us zero evidence.

Then I guess Google could sue her. And they would win.

Looks like she was wrong: https://news.ycombinator.com/item?id=37802116

> Say you search for “children’s clothing.” Google converts it, without your knowledge, to a search for “NIKOLAI-brand kidswear,” making a behind-the-scenes substitution of your actual query with a different query

Wait. Is this example from the court testimony, or something the author made up?

It sounds like the author is massively extrapolating from one slide that was only shown for a brief moment. A slide that we can't see. I am extremely skeptical that Google would make such an egregious substitution and I'd need to see some actual evidence for this.

I'm pretty sure the author is misinterpreting what she saw.

Here is a thing that Google does do: The user searches for "children's clothing". An advertiser has created a search ad for the keyword "NIKOLAI-brand kidswear". If the advertiser has enabled "broad match" for their ad, then their ad will be eligible to be shown on the user's page in one of the advertising slots, even though there are no words in common between the user's query and the advertiser's keyword.

Does anyone have evidence of Google manipulating the user's query to affect the organic search results shown to the user?

Query re-write is probably one of the top keys to search quality, at any search engine.

Is it, though? I reckon it’s commonly accepted. I reckon there are cases that can be highlighted. As I’m sure it’s possible to make the case that some trickle-down policies work. But in the end, plural forms or synonyms do not equate to the original word.

You can just surround a word with double quotes if you need exact matches. It's nice having synonyms otherwise.

Haven't worked reliably in Google since somewhere around 2012 AFAIK.

Edit: Downvote all you want but this is documented.

I use it all the time. My biggest issue is I frequently get 0 search results with it on specific searches. It may not always have a result, but I don't get returned results that don't include the quoted portion.

As I write above it doesn't work reliably.

And as I think everyone on HN knows "my Google" isn't "your Google".

Even if it works reliably for you doesn't mean it does for me and many others.

The problem is that you're assuming search is deterministic.

> The problem is that you're assuming search is deterministic.

I am not assuming search is deterministic.

But even in a generally non-deterministic system, some things should always stay true.

For example: When doublequotes means "exact match", any item that doesn't contain an exact match should not be shown.

I also understand that a webpage might have changed since it was indexed, but I have a really hard time believing all the false matches I have wasted time on over the years relates to websites suddenly changing between the time when they were indexed and the time when I visited them.

Maybe the exact match is in metadata or in hyperlink anchors pointing to the page. More in general, each search will hit thousands of machines and there will be always approximate behavior. It's not just about what you are searching and how, but also what others are searching. The most deterministic aspect of it all might be the latency budgets on the backends and indices. You could tweak those, but then costs, failures and abandonment rates would go up, too.

It used to work as expected before.

It works as expected in competitors with far far far less resources.

The reason they don't fix it is either because they aren't competent to do it or because they don't want to give me correct results.

There is no need to make up excuses for a company the size of Google.

Edit: at this point I should be happy. The faster Google destroys itself, the faster real competition can get a chance and the web can heal.

If you think that teams who find bugs in CPUs or understand some aspects of them better than the engineers at Intel and AMD aren't competent enough to fix verbatim search, I'm not sure what to tell you. It's not about excuses, it's about trade-offs that you might not agree with. For more nuance and history, see e.g. https://research.google.com/people/jeff/Stanford-DL-Nov-2010...

If you believe that this or similar issues will destroy Google, I guess we'll see.

So it is not about incompetence but about wanting to provide a service that is vastly inferior to what they used to do on hardware a fraction as powerful as what they have today.

If what you write is correct they actually want me to have almost as bad results on Google as the ones I get on Bing.

I'll take your word for it.

its usually from user behaviour and how the advertiser setup the keywords for the product. If users searching for "children's clothing" start having a high conversion for "NIKOLAI-brand kidswear" then more and more of their ads will start appearing when searching for children's clothing.

Google (and also the advertiser) have list of keywords that work well for products and then keep testing new keywords

This actually makes sense to me - conversion is possibly the most clear signal that the query intent was answered. While there is a financial incentive for Google to make this happen, it also results in improved user experience - in the less cynical days I might call this a “win win”.

Searching for "children's clothing" gives 100% spam (== store front landing blurbs or things that are labeled "sponsored"). Searching "NIKOLAI-brand kidswear" gives some organic results and some spam.

So, it's now worse than the previous alleged behavior.

Great.

Exactly. “Children’s clothing” is a completely useless search until the search engine joins it with a million signals you failed to type. I can’t even begin to imagine how people believe every page in the internet could possibly be ranked in a useful way without those extra implicit terms.

If it's a useless search, follow the GIGO principle. The user will realise they made a garbage query and will make the keywords more specific.

I find it interesting how you manage to draw such weird conclusions. "Children's clothing" is not a useless search query. It is very generic or wide and THEREFORE useful.

Running that query through a GPT to transform it into "Popular brand among this bucket of users that spends a lot on ads" + "storefront" is completely useless, because now you have "enhanced" the query itself to a narrow subset of indices. All the fashion, sustainability, diy blogs are immediately filtered out in the query itself. The "frontend" now does not even have a chance at drawing from different buckets to form SERPs - every result is storefront now.

This is the issue discussed in OP. There is one implicit question here. Not only how much we need to tweak input queries to get the search engine to actually cast a wide net and allow discoverability, but whether that is possible at all.

Looks like her claim was in fact false: https://news.ycombinator.com/item?id=37802116

This really really doesn't pass the sniff test. It would be extremely obvious if your searches were being so egregiously converted.

>It would be extremely obvious...

I don't think this is necessarily true.

If you search for children's clothing, you get multiple different companies. The query obviously hasn't been modified to just include one brand.

You get companies, but they are all resellers. The listings are all worded like advertisements and stuffed with thumbnails, etc, in the style of banner ads.

None of the results point to manufacturers, reviews or other classes of articles.

Ironically, you get better results with the thing they apparently used to rewrite to.

That's not what I'm saying. I'm saying that I wouldn't expect results manipulated as such to be "extremely obvious" to your average person.

That's what you should be saying, because that's what the article is claiming. If you're arguing they would rewrite queries less obviously, that's a much weaker claim than what the author is making.

I'm sure the average person would notice if their query for "children's clothing" was converted to "NIKOLAI-brand kidswear". Have you ever seen your query converted to such a narrow query targeting a specific brand like the article alleges? Surely you would notice that you're only getting NIKOLAI-brand kidswear results?

>That's what you should be saying, because that's what the article is claiming.

What's with the aggression? All I'm saying is that I don't expect the average end-user to notice something "extremely obvious" like that. Simple as that.

I'm taking no position on whether or not Google is actively doing this, nor am I completely confident that I have or haven't experienced it in spite of noticing very single-brand-heavy results for non-branded queries in the past. I'm only pointing out that the average end-user is a lot less adept at seeing "obvious" things in situations like these than many of us are.

Edit: Put it this way - You and I might notice something, but that's because we're involved in this industry whether directly or tangentially, to varying degrees. I don't expect others to notice something like this if it's not something that they generally care about.

Refute what the article is saying, not your generous interpretation of it. If the author wanted to make such an extraordinary claim, they should have presented extraordinary evidence.

... what are you even talking about? You said...

>It would be extremely obvious if your searches were being so egregiously converted.

... and then all I said was that I don't think it would be so obvious to the average end-user.

That's it. I made no comment on the article, let alone any kind of interpretation of it.

Why are you being so aggressive?

because you're taking the authors ridiculous claim and downgrading it to be reasonable instead of critiquing the original claim, all of this stemming from you taking a quote out of context to make a point that no one was making.

the slice of the quote you responded with:

> It would be extremely obvious [...]

and the full claim with full proper context:

> It would be [...] obvious if your searches were [...] egregiously converted.

the key word being "egregiously", which is a response to the level of severity the author is making in their claim. trying to be an enforcer of conversational tone by repeatedly claiming how responses that you don't want to engage in are "aggressive" isn't conducive to productive discussion. no one's being overtly hostile, outside of you trying to hijack a discussion to exclusively entertain a nonsequitor.

>because you're taking the authors ridiculous claim and downgrading it to be reasonable instead...

At no point have I quoted the article or explicitly suggested that what is said in the article is not taking place.

>... all of this stemming from you taking a quote out of context to make a point that no one was making.

OP made a point clear, that "it would be extremely obvious if your searches were being so egregiously converted". Whether or not I quote the full sentence doesn't change the fact that, no, I do not believe that average end users would find it "extremely obvious [that] ... their searches were being so egregiously converted". And that's literally all I've said here - that I don't think end users would be as perceptive of something like this as OP would think.

>... the key word being "egregiously", which is a response to the level of severity the author is making in their claim.

Whether or not I quote the word "egregiously" does not change my point, nor does it suggest that I've decided to ignore the word and "downgrade" what's said in the article because of it.

I'm struggling to understand how you both seem to be unable to understand something so simple. Feel free to disagree with what I said, but don't twist my words around if you don't like that point.

>trying to be an enforcer of conversational tone by repeatedly claiming how responses that you don't want to engage in are "aggressive" isn't conducive to productive discussion. no one's being overtly hostile, outside of you trying to hijack a discussion to exclusively entertain a nonsequitor.

Responding to me with, "That's what you should be saying," and, "Refute what the article is saying, not your generous interpretation of it," is aggressive, if not rude at the very least. It demonstrates that OP (and you, now) are both missing the point (willfully, it seems, since I have to keep repeating myself over something so simplistic) and are twisting my words, or how I've chosen to quote a passage, around. I know that it's pretty uncouth to trot this out, but my initial comment has a good number of upvotes (so does the comment you just replied to), so there are people out there who have a better understanding of my comment than y'all seem to have.

I am not downgrading anything - I genuinely believe that it's not so obvious to the average end user, no matter how egregious it might be. And that's a refutation of OP's comment, not the article.

https://www.justice.gov/d9/2023-09/416694.pdf

Page 14

"Query Rewriter interprets query"

If this article is based on these four words, I'm not sure how you could label that anything other than a massive misrepresentation.

misrepresentation -> misinformation

According to someone attending the trial, the testimony given today explains what's happening in further detail.

https://twitter.com/megangrA/status/1708892621433672072

That's a tweet by the author of the article being discussed here.

> This system reduces search engine quality for users and drives up advertiser expenses. Google can get away with it because these manipulations are imperceptible to the user and advertiser

I sure have been noticing. At this point finding anything on the web feels like a lost cause.

Besides duckduckgo (bing) which is not even that great either. What other option do we have?

Try Yandex and Kagi. First for breadth, second for depth

How is Yandex, whose stock stopped being traded on the day of the Russian war, is still able to browse the entire Western universe? I’m not expressing a moral opinion, just surprised the West lets them do it.

They are based in EU actually))) Afaik the ru part is a 'subsidiary', at least on paper

None of the (Western) sanctions include widespread internet blocks/filtering, nor would it really be to the West's benefit to do so.

Brave search is really good, its my main search engine, and i found it to be better than kagi with better AI integration.

They are also integrating their own chat ai which is also privacy respecting.

Kagi search is a paid search engine that delivers higher quality results with no ads. I'm currently using the free trial (you get 100 searches for free) and enjoying it.

https://kagi.com/

Also been really enjoying Kagi, just converted to the $10/mo unlimited searches plan even before my trial ran out because it's such a breath of fresh air!

$10/mo seems really reasonable if its actually that much better than google

This also gives you access to some of their AI tools, such as being able to summerize the results of your search using their quick answer feature.

This is something Google can't and won't do because they, and the people they depend on, would lose too much ad revenue. This is going to be a future problem as we transition to AI and AI assisted search for information retrieval.

It's amazing. I searched for a recipe and didn't end up with some bloggers life story.

How does a search engine control this? Did it extract it? Does it search some recipe database instead? I'm confused how using Kagi netted this as this seemed to just be a defect from SEO. Is it just that Kagi doesn't promote pages like that so you end up getting the ones that don't have it?

No, it just had an allrecipes result at the top. Maybe they mark some recipe websites as good? Maybe they put less importance on quantity of text. Maybe they are not motivated to send me to a page where I need to scroll through a blog post (and ads) to get to the ingredients/steps.

One of the things they do is have the number of trackers on a website affect its ranking (negatively). I imagine that is an effective signal in cases like this.

Kagi also uses Bing

According to this page [1], they do use external sources (Google, Mojeek, Yandex and more), but not Bing ;)

They also have their (two) own indexes.

Tbh, for me (as small town dev) it feels like magic how fast those different sources are combined, additionally with user search preferences like raising or blocking certain domains. Insides welcome :)

Edit: Here [2] are some benchmarks and general strategies they pursue. Would love to read more details.

[1] https://help.kagi.com/kagi/search-details/search-sources.htm...

[2] https://help.kagi.com/kagi/search-details/search-speed.html

Crowdsourcing on Reddit, gone are the days when you could answer a question with ‘let me google that for you.’ So you’ll see the same questions appear repeatedly in subreddits as previous answers are lost.

Personally, I maintain an extensive bookmark hierarchy and a collection of rss feeds. I also buy more books instead of reading blogs. medium.com with their subscription baiting articles are PiHoled as the signal to noise ratio is so low that I find it’s a net negative.

I’m also looking forward to locally running a LLM as my own personal search engine.

the SEO folks are starting to pollute reddit threads, too.

The Reddit redesign hid the sidebar, where many subreddits had a FAQ. Now, only that tiny percentage that uses old.reddit.com will see the FAQ, Wiki and all those other treasure troves of information the community had built up. I suspected – and continue to suspect – that Reddit wants the same questions to appear over and over again, as that seems like the very definition of engagement.

I have also seen mods claim that when people post a question, it isn’t necessarily because they couldn’t answer it themselves, but because they’re just seeking interaction with other members of the community. Therefore, old-school net users who refer people to FAQs are cruelly denying them an outlet for socializing.

Browsing with a Pihole really is eye-opening for the "drives up advertiser expenses" bit; I'll Google a thing, click the (very subtle) ad, get a Pihole block, then go down to the actual first result to get to the same spot.

Right? Google target. Top results: ads for target. Halfway down the page? The search result which goes to the same place. Why would I need or want to click the ad if I know exactly where I'm trying to go?

Yandex works VERY WELL for me for general stuff. Goods, reviews, piracy, etc. All super nice.

Things that are location-based do bad, cause they're Russian. They optimize for Russian location stuff.

I noticed the same with the search results in the Google Play Store. When you use search to try to find an app that matches your search term, you almost always get very unrelated apps in the results, and the only good explanation to it is that the unrelated apps that you're seeing provide more revenue to Google in comparison to the actual app that you want.

Or the "unrelated" apps have bought ads on a wide range of keywords so that they appear in many search results?

Like when I search Craigslist for a BMW car and see ads for Mercedes and Audi cars because people posting those ads stick "BMW" in their ad copy. I flag those ads on the basis that they are abusing the search algorithm.

I'd love to say the unrelated apps bought ads, but that's not the case. The Play Store clearly denotes when there's an ad in your search results, but the vast majority of the unrelated apps have no ads.

How much of that is just the vendors putting keywords in their search match dialogs to broaden their scope?

The example I used to hit all the time, was searching for RDP in apple/android/etc app market spaces, the first few hits aren't even capable of RDP and are 3rd parties selling their proprietary screen sharing app rather than actual RDP clients. The app stores seem to have equated RDP or VNC with "screen sharing" and say ~80% of the results don't even support the protocol in question.

Although, without my phone handy I just dumped it into the web based interface and it actually returned a half dozen RDP clients at the top, but then in the second ten, was netflix, facebook, ms teams and various other apps completely unrelated to even screen sharing.

Looks like if you pay enough you can show up in every single query.

Same happens in Apple App Store BTW:

The first result in the results page is never what I search for.

It is possibly just an ad but I can't remember it being market as one and it feels very off in what is supposed to be a premium paid offering as opposed to Googles ad financed ecosystem.

Not defending Google, but this definitely reads like an article written by a lawyer with a vested interest in DuckDuckGo.

Which isnt even independent!

Its crazy to see people advocate for duckduckgo while brave search exists, is more privacy respecting and is actually independent.

Brave sells the shit they scrape for AI training/inference... with no way to opt-out. How is that even remotely close to "privacy respecting"?

So? What does that have to do with anything user privacy related?

As if google and bing don't scrap the web for Ai learning material?

Last time I checked you can block Google and Bing. Brave doesn’t offer this feature.

> Which isnt even independent!

What do you mean?

Who is pulling Duckduckgo's strings?

Duckduckgo is basically bing with another UI.

DDG also do in-line ads like Google, which are scummy as hell.

I use them, but that single factor means I don’t like them.

> while brave search exists, is more privacy respecting and is actually independent

Brave lost goodwill by going crypto. Now that Kagi has a superior product, there isn't a good reason to choose Brave other than not wanting to pay.

A couple of thoughts: Brave started out with crypto, Kagi uses Google's API for results. Brave Search is at least independent and not reliant on Google.

The thing is, at least from my experience, kagi is actually worse, and it isn't independent either.

> a lawyer with a vested interest in DuckDuckGo.

Downvoters obviously do not know who Megan Gray is.

I have never heard of them

https://cyberlaw.stanford.edu/about/people/megan-gray

That Megan Gray?

As per the article itself "[previously] an executive for DuckDuckGo", which seems more relevant than what's on that Stanford page.

I mean, I hadn't heard of them either. But they did state their bias up front in the article, and bloppe's description isn't unfair.

Feels like a good justification as to why the meme of "Google search query results are worse now" might actually be true.

Doesn't that meme also mostly applies to other search engines? Is everyone doing this?

kagi.com and even run-by-a-single-Swede-from his-living-room search.marginalia.nu keeps proving on a daily basis that many of the snags in modern Google is totally avoidable.

We’ve been wondering why query operators no longer work, or why you can’t quote text to require it in results anymore. There have been many explanations. Well, now we know, it would have broken this algorithm, so google dropped it.

The author is making so many assumptions based on a single quote from a slide.

There is zero evidence for what they are accusing Google of doing.

Semantic search doesn't work by replacing words with synonyms. It just uses embeddings where queries using synonyms map to near points in a vector space.

I doubt the actual queries would be manipulated and this author provides no proof other than "trust me bro."

Wired retracted their article: "After careful review of the op-ed, "How Google Alters Search Queries to Get at Your Wallet," and relevant material provided to us following its publication, WIRED editorial leadership has determined that the story does not meet our editorial standards. It has been removed."

I've been trying out Kagi since the HN post from just under two weeks ago. Enjoying it so far.

The post: https://news.ycombinator.com/item?id=37603905

I've been on DDG for a while now. Do you think Kagi is considerable better? If so, how? Genunely interested as I'm on the fence.

I had been using DDG for a while now and sometimes still fell back on Google for searches. I would say that Kagi so far has been as good as the Google of about 2-3 years ago.

The primary reason why I switched over is the argument "nothing is free". I am hoping that paying for a search service means they'll respect my privacy. I'm on a monthly plan so at the first whiff of controversy I'm out.

So far so good, but I'm keeping an eye on it.

> as good as the Google of about 2-3 years ago.

And that's a good thing? That's 2020 Google, in all it's evil glory. That's just around when I switched to DDG.

Ok maybe 10 years ago? When Google was still good.

Kagi has a free tier if you want to check it out.

Noticed the same on Amazon. Searched for Honey Oat Crunch. The first result was honey, then nestle crunch bars. The third result was the Whole Foods brand cereal. Had to click on that listing, scroll to the bottom, scroll through 3 carousels of other products to find the Cascadian Farms brand I wanted.

Then I tried buying the lower priced Whole Foods brand and it had a limit of 3 and a $9.99 delivery fee which offset any savings. Do I drive there to buy it? Is the price the same in store?

At that point, screw it, not going to buy it at all. Too much work just to find what I want. Customer obsession is dead at Amazon.

I really hope that one day we'll get back to a better web. Amazon providing you real options, not extorting you and their sellers. Instagram providing you with content you actually like, not rage and engagement bait. Google showing you interesting and relevant stuff on the web. At this point Idiocracy is not a dystopia anymore I'm afraid. We're pushed to consume bullshit everywhere. Our daily services are dictating our behaviour, instead of enabling us. But I don't know what to do I'm afraid.

People will think I work for the company or am getting a commission or something, but just want to say that Kagi is awesome and a great replacement for google.

Also, in Google's defense, Amazon is obviously doing the same thing. Why is it so hard to search for the lowest price item on Amazon. Obviously Amazon doesn't want you to comparison shop and spend more $'s especially on their ads.

I am wondering if and when this change happened.

I seem to recall the search results got progressively worse over the last few years.

I once used Google Lens to try to identify a weird defect to some grass in my yard. Instead, I got a bunch of results trying to sell me grass seed. This article doesn't tell me much I didn't already expect but it's interesting and hopeful to hear this is coming up in court!

Paywalled: https://12ft.io/proxy?q=https%3A%2F%2Fwww.wired.com%2Fstory%...

This is fucking awesome. 12ft is a hilarious name.

It feels like Google is just running their brand to the ground trying to maximize the profits.

This company cannot die fast enough. Lets all hope OpenAI makes Google Search obsolete.

I don't think replacing one problem with a slightly different problem beholden to the same interests in the end solves anything.

Replacing Google search, with all of its disgusting warts, with OpenAI's chatbot is a dystopian shitscape that I want no part in.

ChatGPT has no citations.

Depends what you use it for.

> ChatGPT has no citations.

I use three programming languages every day, and lookup syntax details often

Using Google (any search engine) returns acres of SEO driven garbage and astro turf

Chat GPT has been a huge boon, essentially no B.S.

No need for citations, the compiler keeps them honest.

I used Bing search, via Skype for a while (it has been degraded recently IMO) it read web pages for me, summerised them, and returned links with it

So I think LLM mediated search is a definite possibility for disrupting the enshitification of Internet search

> No need for citations, the compiler keeps them honest.

But in ldj[...]s preferred future Google is dead and when they are and Microsofts AI only gives you the answers and not the sources they can just keep cranking the prices and there is nothing one can do since you'll never know were they got it.

It could be the cause why Google tries to monetize fast and now, at the expense of extremely fast enshitification.

For most of what I use Google for, OpenAI would not be in any way helpful.

I enjoy searching for opinion/editorial articles on random/obscure topics. I enjoy finding when a niche sport I enjoy gets coverage in mainstream newspapers/broadcasters.

surprised pikachu face

Reply on news.ycombinator.com