Carapace: A multi-shell completion library and binary | Hacker News Reader

Carapace: A multi-shell completion library and binary

carapace.sh

94 pointscab40428 comments

I tried to use this to get better completions with nushell, but it just doesn't have the breadth and depth of completions that fish has as of yet, nor does it have automatic completion generation from manages (although it does have a tool for that) so I eventually gave up and just made fish itself my nushell's completion engine. However I'll be keeping my eye on it, because it's a really cool idea to have a shared completion engine for all shells, so that everyone can pool their work, and because of course having to install two entire shells just to use one of them is a bit silly so eventually I'd like to make the switch.

> so I eventually gave up and just made fish itself my nushell's completion engine

That's an interesting idea. Do you have a link to show how you did this?

Yeah! It's an officially supported thing with nushell, here are the docs: https://www.nushell.sh/cookbook/external_completers.html#fis...

The completions don't work sometimes (for instance you don't get completion for some commands when running them with sudo) and it doesn't fall back on providing file completions if it doesn't know what other completions to provide, which is a bit annoying, but it generally works and I'm sure there are ways to fix that too.

Other similar projects:

https://github.com/sigoden/argc-completions

https://github.com/microsoft/inshellisense

I tried this but couldn't get the completions to load. I think it's a really good idea though (much faster than zsh script completions), so hopefully soon I can retry the install and either get it working or open an issue.

One design aspect I do question is that completions are baked into the library. It'd be neat to have them in SQLite or something that could be updated independently (not sure how feasible that is though. I've never been any good at querying graphs like flag/command completions in SQLite).

Completions need to do arbitrary computation. A command like `git switch` needs to find what branches are available in the repository.

True, but you can represent that in the db as a a CLI invocation to run in a subshell.

The big gain from something like carapace or my theoretical SQLite-based completion system is faster startup time. I had to remove zsh-completions from my shell setup as it added too much to the startup time (https://github.com/bbkane/dotfiles/blob/master/zsh/README_no...)

> much faster than zsh script completions

I'm curious where did you get that from.

Edit: I read the note in your dotfiles repo. Yes calling `compinit` on each shell invocation is going to be really slow. That's not how you're supposed to do it, you could at least add the `-C` flag to cache the completions. Ideally you'd also use `zcompile` to compile the cache to ZSH word code. This puts my completions initializing time at ~20ms on a lower/mid-end laptop. Additionally you can do the trick `fish` does and defer the initialization of completions until the first hit of Tab key, so the impact on shell startup time is exactly 0.

I have had this snippet in my .zshrc for years,

    autoload -U compinit && compinit
    {
      # Compile the completion dump to increase startup speed. Run in background.
      zcompdump="${ZDOTDIR:-$HOME}/.zcompdump"
      if [[ -s "$zcompdump" && (! -s "${zcompdump}.zwc" || "$zcompdump" -nt "${zcompdump}.zwc") ]]; then
        # if zcompdump file exists, and we don't have a compiled version or the
        # dump file is newer than the compiled file
        zcompile "$zcompdump"
      fi
    } &!

Using `zmodload zsh/zprof`, I can see about 50% (16ms) of my start up is compinit (its about 28ms without the `zcompile`).

Do you have any pointers for the "load on tab" idea? I didn't turn up any good results in DDG and LLMs were just hallucinating.

BTW I believe `-C` will disable some cache checking, caching is enabled by default

> To speed up the running of compinit, it can be made to produce a dumped configuration that will be read in on future invocations; this is the default, but can be turned off by calling compinit with the option -D.

> ...

> ... The check performed to see if there are new functions can be omitted by giving the option -C. In this case the dump file will only be created if there isn't one already.

I just want to say - thank you! I've been using ZSH since it became the default on macOS and one thing that started annoying me recently is the slow startup time. Your snippet tangibly improved that.

Do you by chance have any good resources on optimising my config further?

Beyond zprof (https://www.bigbinary.com/blog/zsh-profiling) not really I'm afraid. I did the majority of my zsh-prompt hacking 10 years ago and haven't thought about it since. That snippet could be from anywhere.

You could peek at something like zprezto https://github.com/sorin-ionescu/prezto or pure https://github.com/sindresorhus/pure for tips.

Fetching git/hg/... info is always slow, so try and speed that up where you can (as to how to do that, uhh... I know my prompt has a dirty-state check nicked from pure for speed reasons). You can also cache any `asdf init zsh` or similar to a file and do the same "run in background" trick so the next shell will have any changes.

The biggest improvement I can remember was dropping zprezto for my own much smaller config, I really did not need much comparatively. Mostly some git info and "good default" options. I use zgenom for a plugin manager but only have 3 plugins, probably I should just dump it and inline the plugins to avoid getting owned one day.

You may be interested in this lazy completion loader https://news.ycombinator.com/item?id=40140873

> Do you have any pointers for the "load on tab" idea? I didn't turn up any good results in DDG and LLMs were just hallucinating.

This is bash, not zsh, but I have this working in my dotfiles by just telling bash where to look for my custom on-demand completions:

https://github.com/NateEag/dotfiles/blob/6862726ad2ecaa3a30e...

I imagine something similar works for zsh.

Unable to edit, but this is how I handled lazy loading the completions

    {
      # load compinit and rebind ^I (tab) to expand-or-complete, then compile
      # completions as bytecode if needed.
      lazyload-compinit() {
        autoload -Uz compinit
        # compinit will automatically cache completions to ~/.zcompdump
        compinit
        bindkey "^I" expand-or-complete
        {
          zcompdump="${ZDOTDIR:-$HOME}/.zcompdump"
          # if zcompdump file exists, and we don't have a compiled version or the
          # dump file is newer than the compiled file, update the bytecode.
          if [[ -s "$zcompdump" && (! -s "${zcompdump}.zwc" || "$zcompdump" -nt "${zcompdump}.zwc") ]]; then
            zcompile "$zcompdump"
          fi
        } &!
        # pretend we called this directly, instead of the lazy loader
        zle expand-or-complete
      }
      # mark the function as a zle widget
      zle -N lazyload-compinit
      bindkey "^I" lazyload-compinit
    }

> BTW I believe `-C` will disable some cache checking, caching is enabled by default

You're right, my memory has let me down.

> Do you have any pointers for the "load on tab" idea? I didn't turn up any good results in DDG and LLMs were just hallucinating.

The simplest implementation would be something like

    bindkey ^I init_completions

    init_completions () {
      # ... init logic here ...

      # rebind tab to complete
      bindkey ^I complete-word
      # actually do complete the initial request
      zle complete-word
    }

Edit: I see now you already figured it out, yeah that's exactly what I meant

Completion for program P should be written and maintained by the "owner" of program P - and installed with program P. This is of course difficult when there are many different "shells" that each have their own "language" for specifying completions. A multi-shell completion library can help with this problem.

To me it make sense that completion for program P should be handled by program P itself. That way, completions are unlikely to get out of sync with the application, and the completion handler can use the same option parser as the application. A way to do this is to use a special "hidden" switch to request completion.

Specifically the DomTerm terminal emulator (https://domterm.org) handles its own completions. Bash allows you to register a command that handles completions for some other command. The following tells bash that to handle completions for the domterm command it should call domterm with the magic "#complete-for-bash" option followed by the existing line and position.

    complete -o nospace -C 'domterm "#complete-for-bash" "$COMP_LINE" "$COMP_POINT"' domterm

When domterm is run with "#complete-for-bash" as the first argument (chosen to avoids accidental use) it processes the remaining arguments, and prints out the completion suggestions to bash.

I don't know how to do something similar for other shells; suggestions welcome. Some other shells (such as Fish) include help text as part of the suggested completions. Handling that would require a more complex protocol. Perhaps one that returs a result in JSON format?

If a such a protocol were standardized (perhaps invoked by a "#completions-as-json" option?), we just need to solve a final piece of the puzzle: How does a shell know if a command understands the "#completions-as-json" option? My suggestion is that we define a new keyword-value pair in the "desktop" file specification; the shell would search for a P.desktop file,

> Custom completions can be defined using yaml files.

The concept is awesome but it's curious to see YAML at the core of this project, rather than a Turing-complete, embeddable language like Lua.

YAML is "good enough" for a vast range of tools. It's a good thing that these maintainers are focusing on their core competency/goal of the project and not pandering to niche HN opinions on the stuff that doesn't matter.

Reminds of of Project Hail Mary. Never heard the word "carapace" before that book...

In Romanian "carapace" means shell. It seems to mean the same in English, too, but I presume is seldom used: https://en.wikipedia.org/wiki/Carapace

It's not uncommon in English, though I suspect it's most often used in technical fields like entomology.

I'm pretty sure you encounter in in science fiction as well, where some bug like alien has a carapace or a soldier has some armored exo-skeleton described by the word. Though I cannot think of any specific examples.

Quite common in videogames, too.

A specific example would be the Carapace (armor) upgrades in Starcraft 2 (one faction, the Zerg, are basically insects).

Also sometimes used for (humanoid) armor crafted from shells (fantasy settings).

I think this is MUCH more prevalent in scifi/fantasy targeted at male audiences, and the data confirms it [1]

[1]: https://osf.io/g4xrt/#!

The Dota 2 hero Nyx Assassin (beetle-like creature) has a skill called Spiked Carapace.

Carapace is rarely used outside a zoological context, but people may be more familiar with the derived word "scarab", being a beetle with a notable shell. It is unfortunately unrelated to the delicious dish carpaccio, which is named after a man.

It's pretty commonly used in my experience, but that might just be the types of books I read perhaps.

It's the biggest solid piece of shell in English. Like the stuff over a crabs pincers is still shell but only the stuff that surrounds the main body is the carapace. Often used as an analogy for other sorts of armor, like a knights breastplate being his carapace or a tank's hull armor.

I suspected Brandon Sanderson made it up for his Mistborn series until hearing it in Project Hail Mary also. This is actually why I clicked the thread: was curious if anyone would make the reference!

Reply on news.ycombinator.com