Comment by aboringusername | Hacker News Reader

Comment by aboringusername | Hacker News Reader

Parent Full thread

aboringusername

I think you've done something very interesting that makes me rethink the idea of a "search engine".

For all the data Google has, and all the AI and technology, their search isn't very "smart" at allowing you to contextualize or filter the searches very well.

For example, a button to remove social, to offer meta-data (such as whether a question is answered/unanswered) or offer features that might help me find what I am looking for.

Google has a huge index, but naturally, there's going to be 'noise', and maybe I want to see what's on page 50, or want to try and 'filter' some of the more popular sites.

Their UI for enabling this is non-existent.

Another example: Google BigQuery has the entire Reddit archive, they could surface meta-data such as the number of comments or whether entire comment chains are deleted (although, doing that against Reddit's API at scale would be a very difficult challenge and perhaps not welcomed by reddit).

But to me, as a user of search, that might influence what content I end up viewing, because perhaps a thread with 0 comments isn't so useful to me, or a thread full of [deleted] comments is worthless and should be de-indexed (and deciding what is valuable is a very hard question to answer).

I think you are _radically_ underestimating the cost (and overestimating the benefit) of adding a feature to Google's index.

When you are writing a small-scale search system over a specialized corpus where the index fits in the memory of a single machine it is easy to imagine and realize nifty features.

When your search engine draws 100MW you have a completely different perspective.

You're telling me that google makes $30 for every financial service ad I click but are cheaping out on the signal processing? Is the decline search result quality in the last couple years the result of a business analyst arbitraging sensitivity to result quality vs compute costs?

Only randos on HN think search quality is declining. If the search quality metrics that they use internally suffered a sustained decline, everyone from Sundar down would be fired.

They are declining, and I’ve met many non HN randos who agree. Quality doesn’t correlate with their metrics; likely some sort of mix of “engagement” and mostly how much they make from ads.

Confirm that. A lot of spam and SEO recently.

A whole ton of SEO. Any keyword from which you can make reasonable amounts of money will be cluttered with sites made by SEO people, who are really good at gaming the index, but really bad at anything else (i.e. actually giving proper information on the keyword you're searching for).

Spam has become an issue again in my native language. For a couple of years Google did a great job at removing spam, but somehow in the past 2 years it has resurfaced.

Rando here, subjectively it has been declining for my searches. In aggregate it might have become better so that my aunts' lousy searches are smartly reinterpreted, since el goog manages to "know the person" better than they know themselves. Not everyone wants that!

edit: this is surveillance capitalism at it's finest

Only randos on HN think that businesses don't have a fiduciary responsibility to share holders.

Search quality and profit are not always correlated.

I believe they do have that responsibility. But I also believe that shouldn't be their overriding responsibility.

well i guess that's unfalsifiable, since you're saying this on hn. i have been saying that the search result quality has been getting worse for years... not on every single query, just the long-tail queries that comprise the majority of my searches (guess i'm just an hn weirdo xd).

I don't think that "search quality" as perceived by end users is the metric that Google is trying to optimize - my guess is that their real goal is to maximize revenue from ads. Now if Google's ad revenue were to start declining, that would make their executives fear for their jobs.

Every public statement made on the record by Google search quality people, from Matt Cutts to Udi Manber to low-level engineers' tweets and blogs says that search quality never uses revenue as a metric and they don't generally even speak to the ads side of the business.

Subjectively, my search results have become more cluttered by loosely fitting results than what I'm actually searching for. Even when I'm trying to find a page I've been to before I often need to play keyword roulette and add half a dozen site bans for the likes of Pinterest, bit, quota, and the unpronouncable malware sites Google hasn't purged yet. This is a daily affair. When I'm searching for (generic consumer junk) the result is always available from Amazon,best buy,Wal-Mart,target,and the like. As soon as I stop searching for generic crap Google is worse today than they were years ago because these retailers, review spam sites, etc get boosted to the top of the results despite parameters in the query that should have filtered out these bad results.

In other words. The Google I see has gotten worse at product discovery, and finding relevant debug discussions.

If you feel that the results for your personal subset of queries are getting worse because Google is losing the fight against spam, that is a defensible position and we could discuss it.

The idea that Google intentionally trades away search quality to gain revenue is absurd and pretty much a loonie conspiracy theory.

I just looked at my Google search history and just judging from the last few searches it has been dead accurate. In particular, the result for "mass of a single nitrile glove" is amazing. The top result is an info box of glove mass by size. I don't even know how they do that. Compare the results of the same query on DDG. Those results are garbage.

That is, by the way, how Google defines "search quality". Thousands of paid analysts run searches on Google and Bing and rate the outcome.

Google's losing fight to spammers is a part of the problem.

I'm not arguing that Google is sacrificing quality for revenue. Though I'm sure Google has the run the numbers on such practices.

Instead, I'm arguing that Google search has lost some ability to differentiate between generic and specific. I'm thinking of any of countless cases where I give up on keyword bingo and make a phone call or post in a chat app to get the information I couldn't squeeze out of a Google search.

Google often chops and screws keywords that should not be.

Quoting does not yield expected results, even when I can ultimately find the exact quote on an eventually discovered result.

Fuzzy matches often pollute results for similar but distinct searches. Differentiation is not always easy or even possible.

Fuzzy matches regularly fuzz the wrong words for my searches.

Results often outdated by years or even over a decade.

Limiting by time often returns results outside the range. Mostly in the 'results' before the results.

I often find myself thinking some searches would be easier with a hand written regex.

You touched on what I was driving at in my original comment. There is no doubt whatsoever that if you were able to make a full pass over the corpus with a regular expression you could find docs you can't find on Google's search. But that's obviously not how their search works. They have to make it work at their scale, which dictates the format of the index, which in turn limits the possibilities for query operators. They have to make these design choices so that their product can exist at all.

"Grep the world" is a fine strategy for corpora up to a certain size, and I do wish there was a product that just stored everything I've ever seen and let me run expensive searches on that.

I think this fuzzy logic of search pisses off people like us (the HN demographic) but makes most users happy. Have you seen how tech illiterate people use Google? They type in questions and phrases with words like "a" and "the" in them. This is who google is optimizing for, because there are hundreds, maybe thousands, of people like this for every one of us.

> the decline search result quality in the last couple years

[citation needed]

I think all search engines fail in providing a list of ordered results. What you actually want is a look at the graph that they were plucked from. As things stand, it's not just a case of being unable to see the forest for the trees, but being unable to see the forest because you are forced to infer its structure from a small basket of leaves.

The only company I've seen attempt anything like this is the (criminally under-recognized) carrotsearch.

Do you mean https://search.carrot2.org/?

If he does, I can see why it's not popular. I tried a single query and was told my search limits been exceeded.

While I do not agree with storing user-deleted information, you can change the "reddit" URL to "removeddit" to see most of the comments. Alternatively, "ceddit" will only show mod-deleted comments.

> For all the data Google has, and all the AI and technology, their search isn't very "smart" at allowing you to contextualize or filter the searches very well.

Google exists to make money from 99.9% of users, the ones who don't read HN, so that's pretty normal if you don't find it accurate

Reply on news.ycombinator.com