Ego Graphs – the Google ‘vs’ trick | Hacker News Reader

Ego Graphs – the Google ‘vs’ trick

adsp.ai

366 pointsdatashrimp66 comments

Google used to literally have a feature for this! It was called Google Sets, back when Google Labs was still a thing.

https://www.ghacks.net/wp-content/uploads/2009/08/google_lab...

It lives on as autofill in google sheets: if you write down a set of related terms in a spreadsheet, you can drag from the corner to extend the range using those sets, just like you can extend an arithmetic progression or formula

Didn't that feature originate in Excel?

I think "AutoFill" first appeared in Excel 4 (1992), and was extended in Excel 5.

https://books.google.co.uk/books?id=imoPAQAAMAAJ&q=excel+4.0...

I've found searching the web for things like "when was the fill handle introduced" or "who invented the fill handle" or "what's the history of the fill handle" gives hopeless results.

The parent comment is specifically about Google Sets-style autocomplete of a text list, not just autocomplete in general.

I'm still not sure I get the distinction.

When I try this, it just repeats the items in order, e.g. "New York, London, Tokyo" autofills to "New York, London, Tokyo, New York, London, Tokyo,..."

Do you need to do something special to activate intelligent autofill, e.g. adding "Paris" in my example?

I think it needs 3-4 items to get intelligent autofill?

These instructions don't seem to work.

Alas, that appears to have died way back in 2014 [1].

I remember the first time I used that, however -- it seemed like absolute magic.

[1] http://www.evolvingseo.com/2014/05/20/google-sets-retired-fr...

It seems my memory is not useful when it relates to google products.

Google has the best autocomplete IMO, but below are some other search engine autocomplete endpoints if you want to explore.

DuckDuckGo: https://duckduckgo.com/ac/?q=test

Startpage: https://www.startpage.com/do/suggest?limit=10&lang=english&f...

Qwant: https://api.qwant.com/api/suggest?q=test

dangit. I do this all the time. but if it catches on then it will become useless since it will be over-SEOed.

I feel it already has, with useless suggestions from "alternativeto"-stuff coming up. Even PH spams this.

I really wish there was a way to block certain domains from results - alternativeto, slant, Pinterest, quora and more are just noise and often take up the whole first page of results.

Check out uBlackList: https://github.com/iorate/uBlacklist

Another HN'er referred to it recently and it has been a godsend.

Fantastic addon. Google your own phone number and then repeatedly "Block this site". Search "pinterest" and block away. Rince/repeat.

Great stuff.

Thank you, this is great. I never want to see results from quora, pinterest, facebook, wikipedia, w3schools, .... and these usually occupy much of the first couple of pages of results.

I feel like this ship has sailed. I tried looking for "JobAdder Pricing" a few days ago, and the first 5 pages of results are literally auto-generated bullshit. Feels for the first time in a while like Google are vulnerable to someone coming along with a less shitty search engine (although DDG was no better in this instance).

I use ddg as my primary search engine, and have been doing so for a few months. Unfortunately, at least every 3rd search, especially tech related, I have to switch to Google.

You can add domain blacklists directly to your search queries in google:

google vs. -site:alternativeto.net -site:slant.com -site:pinterest.com -site:quora.com

There's no actual search results being returned, just auto suggest results. Is it possible to game these?

4chan games them all the time, with google then manually removing things from auto suggest (e.g. there are no auto suggest results for "jews are" or "hitler will")

Google actually uses some version of Latent Semantic Analysis which clusters these results based upon word frequency and position. This creates a word matrix where snippets of text can be clustered into themes and the documents are related by a fuzzy concept (document distance in Nth-dimentional space), not directly with key-words like "X relates to Y".

It's harder to "game" that system, but if you know how it works and you're targeting a concept; it'll just force you to create high-quality content so Google wins anyway.

If you really want to play with this type of thing (unlike the blog-post writer that just piggy-backs off of Google smarts) check out the Natural Language Toolkit:

https://www.nltk.org/

I used to be able to do reverse phone lookup by typing a phone number into google. That's long been useless.

What a beautiful way to map out the terrain - I immediately feel inspired to use this as a starting point for hobby-research projects. Only problem is now i'm inclined to check the alternatives to "ego graphs" as a starting point for research!

I have a sneaking suspicion that SEO/keyword mapping with all the resources devoted to that space may have some tools that elaborate on this idea - though im no expert. If anyone knows of useful tools to replicate this in browser I am all ears.

You can call the autocomplete API from JS and then filter and draw the graph. Apparently you don't need an API key and it supports firing requests at the rate of typing.

Neat idea, excellent presentation. What a brilliant write-up, I was thinking "I wonder how this works", and then I got to the detailed explanation of exactly how it works.

This is really useful! I've been exploring some Python data wrangling libraries / Pandas competitors (e.g. modin, Dask, Vaex, RAPIDS) and trying to find a nice way to visualise how they all 'link' together (similar to what Jake does in his "The Python Visualization Landscape" talk[0]).

The linked "Flourish" tool from the article is also really nice. It's a pity it doesn't have an API and is pretty pricey for the premium features, but the default network graph looks nicer than anything else I've seen.

I threw together a Repl here [1] (code at [2]) that lets you put in a keyword and produces CSV output that can be copy-pasted into Flourish (excuse the rough code)

[0] https://youtu.be/FytuB8nFHPQ?t=262

[1] https://google-vs-graphs.garethdwyer1.repl.run/

[2] https://repl.it/@GarethDwyer1/google-vs-graphs

Here’s an interactive version of this idea. https://anvaka.github.io/vs/?query=

[Shameless plug] We have a scalable API for that! https://serpapi.com/google-autocomplete-api

First of all, this is an incredibly cool usage of autocomplete. Super-kudos to the author for finding something so conceptually simple to build, that results in data so rich and useful. (It also makes me miss the old Google Sets even more...)

But the results are so good, it actually gets me wondering if Google's autocomplete results for "vs" are actually just "dumb" statistical text mining from the web, or if Google has special code for when it sees "vs" to look up concepts in its own semantic/knowledge graph and generate the autocomplete out of those?

In other words, are these "ego graphs" distilling what is ultimately textual statistics from the web, or some kind of deep learning model Google has applied over that?

(Separately, I would love if someone could do this for the top million n-grams on the web and build a site out of it... I would visit it SO often.)

I don't believe the graphs that appear here. Trying it myself I obtain a very different thing. For example, from "matlab vs." the second completion is "octave", nowhere to be seen in tfa.

Google results are "bubbled" based on the search profile they have on all of us, but I don't think it makes this technique any less useful.

This google "feature" is really annoying. I have a private firefox window cookies specific for google searches to mitigate this effect. Yet it is still there.

While the visualizations are cool, I'm a bit afraid of how much of the displayed word cloud is a visualization of the "bubble" that the author lives in. It would be cool to see how different these graphs are on each of our computers!

Google has a URL parameter to turn the personal search feature off.

Sure, but it does not really work. It still tries to infer your search "preferences" from your IP, and whatnot.

And there is a reason why the bubble is still there even in private browsing..

https://www.vice.com/en_us/article/m7jvvp/google-sued-tracki...

It looks to me like that graph was produced using k=3 for the k_edge_subgraphs() pruning step, and likely the suggestions for "octave" only included one or two other ML-related terms (because they are crowded out by musical terms).

(For me the "octave vs" query gives "matlab", "Python", "pitch", "rhythm discord" and "key" as the first 5 suggestions).

I found "octave" is a direct connection of matlab using @anvaka's very cool tool: https://anvaka.github.io/vs/?query=matlab%20

Can someone point me to resources on how to re-implement the "vs" feature?

I feel like this can be done with just GPT-2 instead of using the google search api. For example if you type something like "Instead of using Tensorflow, use ..." into https://talktotransformer.com/ it can babble pretty coherently about related technologies so the data is there.

I have a little paper on this, sorry for the plug :-D

http://ingomarquart.de/index.php/research/2-the-semantics-of...

uses BERT, but GPT2 would work as well. I am working on making this a python package and a technical article on the method which works really great. One example is that I can use clustering to distinguish out syntactic information and then get a purely semantic ego graph (as used in the paper for roles).

I assume they have similar for “convert to” though I always loved the idea of pdf as an alternate religion :)

FYI clicking the link redirected me to a medium link. Is there a non-medium link for this article?

Really cool idea! I didn't find the chess openings graph particularly useful, but what a fantastic way of finding related media! I'd love to generate clusters for my favorite movies, music, and games and use it for new recommendations.

> Buy a dog!

Don't buy a dog. Adopt one.

I decided to try this with my last name, taking the ego graph concept literally. Along the way I discovered the existence of MMA fighter David Hošek and the Hošek-Wilkie model for sky coloration.

I tried it with my online name, tzs, and everything the vs. trick gave was a currency code. It turns out "TZS" is the symbol for the Tanzanian Shilling.

I had no idea.

I wonder if that is why I get occasional inquiries about buying my domain?

would be nice if this were a hosted service or open source container!

Agreed! Would be a great weekend project for a junior developer or someone trying to break into the industry.

All the steps are there, just not the packaging.

Hmmm. I turned off the autocomplete so long ago that I don’t remember how to turn it on. This is the first reason I’ve wanted to since when.

I am still confused by the color of the nodes, seems arbitrary (and hence: confusing) to me.

This is very useful. Thank you!

This is fantastic. It takes the idea of "vs" queries and adds a graph to glue together the various "vs" relationships. Props.

I only have one nitpick. The graph with dogs didn't include boxers (which everyone knows are the coolest dogs).

Well the graph starts from poodle, which is pretty far from boxer. I'm sad Cavalier King Charles Spaniel aren't there though

I was just kidding. But you make a good point about the starting point which makes me think of an obvious extension: iterate "vs" operation for every node and keep adding nodes until it stabilizes. The set of dogs is finite so the graph must stabilize at some point. If I do "vs" for pitbull then I get boxer and I see pitbull in the graph.

Welp, now Google will lock this API down soon.

How would they, without impairing normal users? You need an API for the public that allows you to get suggestions rather quickly without an API key, for all the users that are not logged into their Google account.

Probably the same way they lock down Google search from too many HTTP requests.

Captcha? This won't work for their regular use cases.

Probably server-side rate limiting based on IP.

Yeah I'm sure there's a reasonable flood control on that endpoint, which for project like this probably won't kick in.

This API has been well-known forever.

This API is hit on every search keystroke in modern browsers with a small debounce delay, so its tolerance to frequent requests will be pretty high.

Apple -> Android

Google -> Bing

Microsoft -> Amazon

Twitter -> Trump

Messenger -> Whatsapp

Facebook -> Instagram

Instagram -> Reality

Reality -> Dreaming

Dreaming -> Awake

Awake -> Asleep

Asleep -> Dead

Dead -> Alive

Reply on news.ycombinator.com