Hacker News Reader

Comment by mschrage | Hacker News Reader

The TS/JS analogy is perfect. This is exactly the approach we took with Fig.

We defined a declarative standard that lets you describe the structure of any CLI tool. Fig then uses these to provide VSCode-style autocomplete in the terminal!

Currently specs are manually contributed by the community, but we are planning to integrate with CLI libraries (cobra, commander, click, etc) so they can be generated automatically!

You can see some examples here: https://github.com/withfig/autocomplete

View on HN

Fig seems to be reinventing the wheel. Shells already have their own completion systems to handle exactly this problem, and you can change their default frontend to something like fzf. (See, e.g., fzf-tab.)

While they exist, shell based autocomplete have always felt like a suboptimal solution we just accept out of historical happenstance.

- You need to press tab to get any completion to display

- You don't know what completion will appear when you press tab, or at most one completion displays

- The completions are not at all aware of context, at best just doing some untyped fuzzy matching. Compare this to Fig, which at least seems to know what sorts of arguments each command accepts and displays a list of them

- They only provide completion, not inline documentation, which is one of the big points of the article

Most of those things aren't true of fish:

- It autosuggests on the command line without typing TAB

- The completion is programmable, so git completion knows where a branch is expected or a path is expected, etc.

- When you hit TAB, you get descriptions of each completion

None of these are true.

- You can get the completions to display as you type (see zsh-autosuggestions, zsh-autocomplete).

- Completions are completely aware of their context, probably more so than what fig can infer. You do need to actually load the completion functions of your binaries though, which are traditionally named ‘_command’ (so, an example would be ‘_git’).

- They can provide documentation; fzf-tab does so.

Ouch. I suggest you learn to configure your autocomplete. Pretty much everything you say is incorrect except perhaps the last point. I've used context aware completion and with multiple options shown for over a decade!

> You need to press tab to get any completion to display

You only need to do that if you want to bring focus to the completion selection. A lot of interfaces will display completions without the need to press tab. And to be honest, while I'm all for making things simpler, I don't think pressing tab is a hard barrier to expect people to overcome.

> You don't know what completion will appear when you press tab, or at most one completion displays

This doesn't make a whole lot of sense. If you know what completion is going to appear then you're hitting tab to save keystrokes. If you don't know the command then of course you're not going to know what completions will display since the whole point of hitting tab is to explore the valid options.

> The completions are not at all aware of context...

That's not even remotely true of Bash, let alone any modern shells like murex and fish.

Murex even goes one step further and doesn't just display the parameters in context of the command that's being run (eg suggesting names of available branches when running `git checkout <tab>`) but it runs the entire command line that precedes it to understand the data being passed into the STDIN of the current command. This is useful when using tools that inspect JSON properties for example:

    murex-dev»  open https://api.github.com/repos/lmorg/murex/issues | [[  ]]                 
    (builtin) Outputs an element from a nested structure                                                                                      
     /0                                /0/active_lock_reason             /0/assignee                       /0/assignees                     
     /0/author_association             /0/body                           /0/closed_at                      /0/comments                      
     /0/comments_url                   /0/created_at                     /0/events_url                     /0/html_url                      
     /0/id                             /0/labels                         /0/labels/0                       /0/labels/0/color                
     /0/labels/0/default               /0/labels/0/description           /0/labels/0/id                    /0/labels/0/name                 
     /0/labels/0/node_id               /0/labels/0/url                   /0/labels_url                     /0/locked

> They only provide completion, not inline documentation, which is one of the big points of the article

Nope. In murex if you type `kill <tab>` you'll get a list of process names instead of PIDs and when you select one it is still the PID that is placed.

    murex-dev» kill
    (/bin/kill) kill - terminate or signal a process
     543     /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code...
     738     /usr/libexec/promotedcontentd
     1645    /System/Library/PrivateFrameworks/DifferentialPrivacy.framework/XPCServices/DPSubmissionService.xpc/Con...
     17903   /Applications/Slack.app/Contents/Frameworks/Electron Framework.framework/Helpers/chrome_crashpad_handle...
     47968   /System/Library/Frameworks/Metal.framework/Versions/A/XPCServices/MTLCompilerService.xpc/Contents/MacOS...
     496     /Applications/iTerm.app/Contents/XPCServices/pidinfo.xpc/Contents/MacOS/pidinfo

Likewise if you type `git <tab>` you will get a list of all the next commands that follow and what they do:

    murex-dev» git
    (/usr/bin/git) git - the stupid content tracker
     init           Create an empty Git repository or reinitialize an existing one
     restore        Restore working tree files
     revert         Revert some existing commits
     submodule      Initialize, update or inspect submodules
     push           Update remote refs along with associated objects

The output is colourised and highlighted so it makes more sense in a terminal than it might appear in here. But you get the idea. And all of the suggestions are scrollable with the cursor keys, you can quickly jump by typing more characters, or search for specific completions using regex if you press ctrl+f (this last feature also makes it very quick to traverse large directory structures in `cd`)

Yeah, we don't want this to be a '15th standard' situation [0]

The biggest issue with traditional shell completions is that they are kind of tricky to build. Everything is defined imperatively and written as a shell script.

We wanted to lower the barrier to building completions and make a representation that works across different shells.

[0] https://xkcd.com/927/

How far have you gotten in making it work across different shells? I'd be interested in adding something to https://www.oilshell.org/, and have been thinking about this for awhile (see sibling comment in this thread)

A big problem I came across is that any shell-agnostic solution is likely to be hard/impossible to implement in bash, which is the most popular shell :)

Since Oil already emulates bash, it was easier just to implement bash APIs and get a huge corpus of completions for free, rather than try to develop a new corpus.

It looks like Fig does things in the terminal emulator / OS and not the shell, which is interesting. Does an approach like that work on Linux?

Another problem I ran into is that GNU readline is pretty limited as far as the UI goes. Fish and other shells have a better UIs but they are coupled pretty tightly to the shell.

First off, I've been a big fan of Oil shell for a while! Love the blog. :)

Fig currently works with bash, zsh and fish. Since we are operating at the OS level rather than in the shell, we can get around the inflexibility of readline/bash.

In theory, this approach would work anywhere. What we do currently on macOS is especially 'involved' since we need to provide the completions UI in a separate GUI app that we don't control. But, in general, integrating at the OS/application level should be possible on Linux and Window as well.

Great, yes doing it at the OS level is a creative approach and I can definitely see advantages to that. I would be worried about having to parse zsh and fish as well as bash, although I suppose coming up with something that works most of the time is feasible.

As mentioned elsewhere I wanted to have some invariants for correctness, but maybe not all of them were necessary. I felt it was useful to separate the problem into completing the shell language vs. completing the argv.

Feel free to join the Oil's Zulip channel (link on home page https://www.oilshell.org/)! There are the past discussions on #shell-autocompletion, going back a couple years. It's dormant now, but as mentioned, there were multiple people who wrote code towards this. And there were debates around the issues above.

Another interesting channel that just started is #shell-gui. Short summary which I have yet to blog about: I just implemented "headless mode" for Oil (analogy to headless Chrome). So I can punt the UI for the shell to multiple other projects :) As I mentioned a few times, I realized the scope of the project is too big. I have a collaborator Subhav who has a prototype of a GUI in Go (and shell).

Basically the shell a language- and text-oriented interface, but it does NOT have to be terminal-oriented interface. It took me awhile to realize that we shouldn't conflate those things! And it was pretty easy to tease them apart in Oil. The headless mode provides a simple interface and allows integration that can't be done with bash (or any other shell AFAIK).

It needs feedback from people who want to build GUIs. An easy analogy is to imagine is a browser-like GUI for a shell with the URL bar as the prompt. The URL bar provides autocompletion, history, and shows state, etc. just like shell does.

Just joined! Also love the idea of a headless shell. Excited to chat more

Just the other day I added completion support for our cli build with cobra and it was <10m work.

So I’m not convinced this is still a hard thing to do. (The ui of fig does look slick though.)

Glad you like the UI!

Cobra and other CLI libraries, like oclif, can help you generate the skeleton of the CLI, stuff like subcommands, options, etc. (I'm in the process of writing the integration so you can generate a Fig completion spec the same way!)

The difference is that with Fig you can add richer completions as well. For instance, Fig's completion for `npm install` allows you to search across all npm packages: https://twitter.com/fig/status/1385401292193865731

Cobra has also support for completion. Through defining a ValidArgsFunction. This seems the same level as the completion shown for npm install.

Woah, that's really cool. This seems to check the first two of the article's points, approachability and discoverability. Are there any plans to implement the third point, interactivity, where you have small, dynamic guis embedded in the terminal?

The core focus right now is autocomplete, but this is definitely an area that we want to explore more in the future!

Reply on news.ycombinator.com