Hacker News Reader

Comment by ramoz | Hacker News Reader

When I search “tim lee food blogger age” Google actually shows results with “age” striked out (so it shows top results as if age wasn’t part of the queried string).

Trying to think why/how it’d conclude that age wasn’t necessary for good results.

View on HN

Query rewriting, a useless and use-hostile technique, strikes again! (No I am not talking about trying to fix spelling mistakes with "did you mean". I mean straight up removal or rewriting of correct English words)

I am embarrassed by the whole field of NLP for making such a big deal out of a task that is fundamentally bad.

Google optimizing for the "average user" comes at the expense of the whole world, because the "power users" who are optimized against literally build the internet for the normies. Cater to the normies for too long, and we see the status quo.

Force the normies to get better at writing their queries. NLP will unironically have a large number of people with the title "query engineer" soon anyway due to the rise of increasingly large foundation language models proliferating. Welcome to the brace new world of botched semantic search!

> Google optimizing for the "average user" comes at the expense of the whole world, because the "power users" who are optimized against literally build the internet for the normies. Cater to the normies for too long, and we see the status quo.

If I were feeling cynical, I would point out that catering to us was a great idea for early Google, but not so much now. The problem with power users is that the more control you give them, the more they learn about how your product works and what decisions you're making. Power users can pick out dark UI and other user-hostile patterns more easily.

Early Google needed the power users, because we were an essential component of building Google's early dominance: It was us gesturing the normies over and saying "as the computer person (TM) in your life, you should be using Google. They're cleaner and their results are better." We were needed to both scale their user base and plug gaps since at the time there weren't dozens of engineers working on issues and search + web indexing was still in its infancy.

Now, though? There's no benefit to Google in engaging with power users. They have the numbers of normies they need for profit, and all engaging power users would do now is lead to more conversations like this in the real world, which is not what they want.

"Oh, you're still using Google? They suck now. Use X, Y, and Z." = conversations Google doesn't want.

We cater to power users. If you absolutely, positively need a word or phrase to be on a page, use quotes. "age" or "tim lee" would tell us that exact word or that exact phrase has to be on the page. Which still doesn't help with this query, given that his age doesn't appear to be known, but you can do it for others.

Even quotes don't actually work. I've witnessed google query rewrite even with full quotes. I've also seen other posts on HN from others who've had this happen to them.

Quotes do work. They really, really do. Every time I look into a complaint about this, I find that the text really is on the page. It just might not be readily visible. Twitter link is in my bio here. If you ever get an example where you find quotes don't seem to be working, do pass it on.

We show this to indicate to you that we found a page that might be relevant but which doesn't contain that exact word. And why we would conclude that is explained better if you search for ["tim lee" food vlogger "age"].

That's telling us to find only pages that have "tim lee" on it in that order, as well as the word "age" and food vlogger are nice to have but not required. And it turns out -- there's not a lot of pages. Certainly not many pages, it seems, about the Tim Lee you're looking for with his age. And that's because not everything we want is actually out there. There might not be a page that has his age.

So back to why we dropped it. By showing you some pages that don't have the word age on it, we're able to show some other pages that are generally about him, which might get you closer to the answer.

BTW, there is a Wikipedia page I've seen suggested as having the right answer for his age. But that's for a different Tim Lee -- not the food vlogger, and it also only lists his birth year, so knowing the exact age is hard

Thanks for the follow up, I realized all of your points afterwards.

It no longer does this because now your comment is the only result for the query. Before, it could not find a single result, said as much, and gave you the wikipedia page that included the age. What's wrong with that, again?

I would prefer that they just returned 0 results and had a list of links for alternative queries.

No doubt that’s bad for some opaque internal metric though.

I realized after testing some more, that the query might not have had many relevant search results. It’s still a strange tactic. Age is contextually important in my query - Google didn’t recognize this

Censoring terms of the search vector to find relevant results is a legitimate approach that works well. In this example, the article author gives a Wikipedia page as the desired top result. That document does not include the term "age" anywhere. Therefore you can see how the term is not necessary to the search.

ty. I also tested the query in Bing, the results there were abismal

It works like trash when searching for obscure technical problems, e.g. debugging some little used tool in linux. It seems more often than not the most important word in the search is omitted, making the results completely irrelevant.

As a user, I can’t think of a time when censoring a term has given me relevant results. It just serves to frustrate me. If Google doesn’t want to return a blank page, they ought to have a small section at the top that says something like, “No results found for [search terms]” and then another section following that clearly states it is excluding a term.

FaceBook Marketplace and Craigslist do something similar when there are no local results. They show results from outside your area but clearly call it out.

Could you all stop misusing the word "censoring" like that? It's almost worse than the way it's usually misused.

Also, at least for me, Google does exactly what you describe at the right approach.

While I agree this isn't technically "censorship" from one perspective, I do think it's important to consider this: by forcibly omitting search terms against your wishes, it is actively and deliberately preventing you from searching from what you wanted to search for. This is both frustrating for the user, and has a detrimental effect upon the quality of the search results.

When most people search, they're generally looking for information that matches a topic, not necessarily every single word in a query. You can imagine the number of misses that would happen if we didn't compensate for misspellings, for example. Or for synonyms or plural forms. But in no way are people prevented from searching for exactly the words indicated, if they feel that's best. Use Tools > All Results > Verbatim from the search toolbar. Or put quotes around the word or words you want exactly sought.

I'm not misusing it. Dropping dimensions from a document term vector for similarity computation is called censoring.

We wouldn't say the word is censored. It's just an indication we found a page tha that might be relevant even though it doesn't contain one or more of the search terms, and we want to help the user understand that. But yes overall, that's exactly why the approach can be useful, in case a page with the right information uses a slightly different related word. Also, that Wikipedia page is for a different Tim Lee -- a comedian, not the food vlogger.

Reply on news.ycombinator.com