Hacker News Reader

Comment by gregmac | Hacker News Reader

> if you're searching for [camping sites near silicon valley] and [rust http libraries] and then try to compute a similarity metric that includes both of those lines of querying, you'll get wildly different results than if you separate them out and then compute similar sites.

Wow, never thought about how complex of a problem that can really be. To hijack your example: searching for [rust http library] and [clean rust from metal bookshelf] should be completely separate lines of querying but without understanding the full context, appear to have many false similarities (rust, library/bookshelf, 'clean' is a word associated with compiling software, and 'metal' can cause confusing with projects/articles on [bare metal programming with rust])

View on HN

Yeah, disambiguation is a huge problem in search as well. I remember that when I joined (2009), I kept saying we need better disambiguation algorithms, because I was working on UI stuff and getting nonsensical results when I hooked up other teams' backends to the search result page. A bunch of teams were formed and I think the problem was largely solved by 2012 or so - there're some clever tricks you can pull when you have the amount of data and computing power that Google does.

Google drives me absolutely nuts because of it's 'intelligence', if you will. With dumber search engines it is trivial to ban words you would only expect to see on pages unrelated to your search phrase, using '-' or something similar. Similarly, if there is a word I know is associated with another topic, but is unlikely to appear in another topic, it's easy enough to force it to appear using '+'.

Google's methods for detecting clustering, or whatever they are using, are so 'clever' I feel like I'm unable to communicate what I want. I feel like I'm arguing with a particularly intelligent and malicious toddler sometimes.

Say I was to ask a toddler to bring me a beer, and they brought me a root beer. I'd be like, "no, bring me a beer, -root."

A normal child would bring me a normal beer, cause that's the only thing in the fridge that has beer in the title without root. The Google child would bring me some sassafras, because sassafras is a root beer, which is a beer, but it doesn't have root in the name.

Annoyed, I turn off safe search, and I say "Bring me a beer, -root -sassafras -sarsaparilla -ginger -cider."

The Google child comes back and says "I couldn't find what you were looking for." I get up off the couch, look in the fridge, and lo and behold it's completely full of beer.

You're looking for Verbatim Mode:

https://www.google.com/search?q=beer&tbs=li:1

this. Thank you!

Awesome! Is there a way I can make this the default?

You can't do it for all Google, but if you use Chrome, you can set it as the Omnibox default. Go to Menu -> Settings -> Search -> Manage Search Engines, then add a new row at the bottom that says "Google Verbatim", "verbatim", "https://www.google.com/search?tbs=li:1&q=%s", hit Done and click "Manage Search Engines" again, then find that row in the list above (alphabetical) and click "Make default". Now everything in the address bar uses verbatim search.

wow, you just fixed everything that i thought was wrong with google. thank you.

I've found that part of the solution is to give it more to work with, and in a 'dumber' way. Remember, Google search isn't really a query language any more, it's more of an NLP find-what-I-mean interface. So sometimes instead of giving it keywords and logical operations ('beer +fridge -root -sassafras'), you can get better results by just asking a question ('get one of my beers from the fridge').

Are there any that you are at liberty of talking about? I find machine learning especially applied to search and NLP a really interesting problem.

Probably not - when it comes to search algorithms it starts to get dangerously close to things Google considers very confidential.

Machine learning is definitely a big part of it, but you can also get surprisingly far with simple bag-of-words models when applied to enough data.

Reply on news.ycombinator.com