Bash-LSP: A language server for Bash
github.com
github.com
This is built on tree-sitter, but there are a couple of other interesting shell parser projects:
* https://github.com/mvdan/sh
* https://github.com/koalaman/shellcheck
At Fig (https://withfig.com), we had to write a bash parser from scratch. These projects all work well in the IDE context, but in a repl environment (ie when typing in your terminal) they don't support incomplete commands very well. e.g if I type `echo "$(` the above parsers will break as the double quote and parentheses are not closed.Tree-sitter is more general: it aims to provide support for many languages and so its bash support is lacking a little. But it's very fast. It's designed for IDEs (hence why bash-lsp is using it). It caches its results and only re-parses on newly changed code blocks.
sh by mvdan and shellcheck are both built specifically for the shell. They both have really robust grammar. shellcheck is great for error handling. sh is great for building ASTs (it's more of a true parser)
As always, it depends on your use case. If you need a descriptive summary of errors / warnings in a shell script and performance isn't crucial, use shellcheck. If you need a really robust parser and AST output, use sh. If performance is crucial and/or you eventually want to support different languages, use tree-sitter.
Please feel free to jump in if I am wrong here!
* scripts being poorly documented: what's the point of having a script if no one knows how to use it
* scripts going stale: often built by one person / an infra team. When they break, no one knows how to fix them
* creating scripts: bash isn't the most popular language these days, but it's still supported just about everywhere. How can we make building them easier and faster but still just as powerfulConsider my problem: I use `zsh` and have created, collected and organized a large number of completions which I synchronize among several workstations. It frustrates me to no end that I cannot take advantage of these completions in very many other convenient places, like VSCode[0]. The benefit of writing it in `zsh` (I can't speak for bash) is that seems like it'd be pretty simple to wire up the LSB as a sort of "completions proxy" that simply takes the inputs, passes them to a completion script, parses the output and formats it in the appropriate JSON response. The utility of being able to have a single script (or set of scripts) that I can include in my shared profile directory which will run and handle a `zsh` LSB implementation on-demand (i.e. doesn't need installing, sits in my scripts directory and runs everywhere) would be a huge feature.
This will speak to my ignorance, but can you host a web server (in a manner that can be consumed) from just bash and a lean environment[1]? Could you host an LSP without one (i.e. is there enough to use a socket and would it be enough to point it at the right place/scheme?). I believe there are `zsh` extensions which can facilitate some/all of this, but I'm not sure if it would work out of the box. The downside to developing it in `zsh`, other than the language itself[2] is performance. Both of those, assuming "the hosting/serving" can be handled easily, don't represent big problems, maybe they are. But performance and the language should be up to what remains outside of that.
[0] I had `nvim` set up correctly, once to work with completion scripts and was able to translate that into VSCode doing the same, about two years ago. I'm still not fully convinced I didn't just dream this up because the few times I've looked into it, since, I've gotten nowhere (admittedly, I've put little effort into researching).
[1] I'm thinking 'busybox', minimal gnu-related things, ideally requiring as little as possible beyond a relatively recent version of `bash` so that "if it's got bash, I can use it".
[2] I use `zsh` because it's easier for me to write things in it than `bash`, and at times it can be fun. But there's a reason we don't build solutions using it.
[3] Everything's a string...
The developer experience out of the box with TypeScript + VSCode + Next.js is 1000x better than any stack I’ve ever worked with in Python, for example. So what if you need to pull in some dependencies? At least I don’t need to figure out what the best way to setup a Python environment is this month.
(In fact, the current Python package manager du juor, Poetry, took most of its ideas from what Yarn implemented five years ago.)
And what’s this about being irresponsible? You can easily pin your dependencies, use a lockfile, and even bake the cache into your Docker images. There’s no reason you can’t have a program that will run the same in two years as it does now. And setting that up is a lot easier than it is with other languages.
You should practice your April fools jokes to be less obvious, that post is not gonna make it.
I love the idea of frontend dev. But the tooling takes all of the fun out of it.
And I agree. Unless it works out of the box, it's a half-day of profanity at least.
But writing a command-line tool in TypeScript doesn't require any of that. The comment focused on FE development with Next.js -- I have no experience with that, but a simple React-based solution using `create-react-app` works out-of-the-box for a lot of things[0]. I will say, it does sometimes feel like everything TypeScript gets right is made up for by everything "all of the bundlers/tooling around it" screws up, but it's gotten monumentally better. You're always going to be weighing pain points against eachother, and folks that do front-end web development all day long are going to find the problems they routinely encounter to be "simple", possibly even "intuitive" based on how the rest of the ecosystem works.
From my perspective -- partly on the inside, partly on the outside -- I seem to have to muck around with the equivalent of "build scripts" about as often as I used to (not) enjoy in the C/C++-land. But dammit-all if "time to completely working" in TypeScript isn't always twice as quick. TypeScript, configured correctly, catches so many run-time issues at compile-time[1]. For me, the distance between "it runs" and "it runs correctly" is often shorter in TypeScript despite it not being my primary language.
[0] To be sure, it's hard for me to say `create-react-app` without the obligatory `fscking` inserted... it's got many problems, but at least half the time it's an implementation detail.
[1] Static-type-checking time?
The reason I don't write everything in TypeScript has nothing to do with TypeScript. Language-wise, it is my favorite experience, hands down. I can express things in a type-safe, strict manner, and do things generically that simply cannot be done outside of dynamic languages, while still gaining the benefits of a statically typed language.
Not taking node into account[0], TypeScript on its own is my favorite language and it's not the one I write software in every day. While I agree with your arguments around pinning and such, node is a bit of a victim of its own success and suffers from dependency-hell in a way that is truly unique and interesting all on its own. The only way to be certain that code is good is if you write it, barring that ... a trusted, supported third-party for the important things, and inspect the rest. The problem on the node side is the impossibility of the last task, and if you aren't doing it, neither is your trusted third-party. I'm not saying it's hopeless -- I'd even wager a guess there's a comment in my past that is far more harsh on `npm` than it deserved/deserves[1].
I don't think "node-based" solutions for shell are bad. The biggest problem I tend to run into with shell scripts in constrained environments are related to dependencies. With node, `install`, done (usually). I target "what's probably there" like most people. Sometimes it's not. In the past, when this was a bigger problem than it is now for me, I would throw my hands up and build it in whatever I wanted because "if I had to get a waiver to add `jq` or some other thing to this box, I might as well have them put `python` on there" -- same amount of work arguing over BS but less work writing it. I'd probably apply that to `node` these days but those problems aren't mine any longer. :)
[0] Which some will point to `npm` as the whole reason they use node while others will point out why it's the entirety of its problems.
[1] And I hate the attitude of some that implies `npm`s problems have easy solutions, "if they just ...". No. It's almost everyone's first language. It's really easy to publish packages. There's a gajillion of them out there. (Yet, it works ... every? ... time I need to solve something using it.)
Also, lots of Linux distributions don't come with GCC and Clang by default (Ubuntu, Debian, OpenSUSE).
Python is also the 2nd most popular scripting language in the Linux world; it's even part of the Freedesktop base runtime. Though I don't know if it's standard in BSDs.
Most distros don't preinstall C compilers.
Care to elaborate why?
Not particularly, since it'll only fall on deaf ears considering HN's demographic.
This line right here means there's no runtime dependencies https://github.com/bash-lsp/bash-language-server/blob/fa5a65...
edit: turns out there are runtime dependencies (see child comments). Still, typescript is not one of them.
And either way, you don't need TypeScript to run this language server. You just need node. If you're not happy with that, well, consider this repo a reference and get cracking.
https://github.com/bash-lsp/bash-language-server/blob/fa5a65...
When I gave this a shot a while back, I looked at its dependencies and it was way larger than the footprint of coc-clangd for some reason.
Wasn't a good signal so decided to remove it after that.
software is so modular nowadays that you can write language-server in C#, for an Typescript IDE/Editor that helps you to write Bash scripts for Linux.
Doesn't make any sense. LSP is just a protocol. You can write language servers in whatever language you want.
They took a Bash parser written in C from the Tree-sitter project, and are using the C as a compiled WebAssembly blob out of TypeScript.
And, actually, that Tree-sitter Bash parser is specified or generated using JS somehow; the project for producing it uses TypeScript also.
That's how easy it is to write LSP server in any language?
Maybe it's done this way so that the LSP can be integrated right into the editor itself, so as not to have to run somewhere else as an external piece.
Personally I would have used Go for this exact reason, but it's reasonable to use TypeScript too.
Not true. This is highly interchangeable tooling - it can have whatever dependencies you want, because there's no "dependency explosion" that happens with libraries. LSP is a standardized protocol - you can swap it out with anything you like.
Moreover, people rarely build interactive tools on top of other interactive tools. Now, if this were a library with a bunch of dependencies, that would be a whole different matter - then any program or library that used this would get all of the transitive dependencies - but, it's not.
Go and reimplement this tool yourself. RIIR, make a statically-linked 500kb executable with no perceptual execution time, and eat their lunch, because your tool is a drop-in replacement.
Is a Node dependency a bad idea for this tool? Arguably yes. Is it "irresponsible for the future of software development"? That statement is so false that I suspect it's being used to try to emotionally manipulate readers instead of making a coherent argument.
Also...are you aware that Bash already tab-completion, even if it's not automatic?
I'm beginning to wonder if this is a very subtle April Fool's post...
It's worse than you think.
This uses something called Tree-sitter: some project written in C for providing parsers for numerous languages.
The Tree-sitter parser for Bash itself somehow requires Typescript, though the parser is in C.
This Bash-LSP project uses a binary blob of that parser: it is C compiled as C++ into WebAssembly.
Bash-LSP, as well as the Tree-sitter Bash parser, can basically be called IDE tooling written by and for TypeScript programmers, who sometimes have to mess with Bash scripts.
The target audience doesn't mind TypeScript and Node dependencies, and in fact likely finds that preferrable to any other kind of dependency, because working with TypeScript dependencies is fresh in their minds, requiring little ramp-up to handle another one.
This is not necessarily true if you can write a compiler for a tree-sitter grammar that outputs the parser code in your favorite language. I know semgrep does this to make parsers for OCaml. But, of course, that's more work to be done.
You can use tree-sitter to produce nothing more than C parser file that only depends on a very small tree-sitter C library.
So you can use tree-sitter to produce a parser that does not require Typescript.
However, there is whole other side to tree-sitter which is about filtering/manipulating/searching the AST results produced by that C parser and this layer has binding to many languages.
I suspect this Bash LSP project is using the tree-sitter Typescript bindings for their AST processing.
Could anyone with more knowledge please explain what this is and how one might utilize it? Fwiw, I’m deeply familiar with Bash; I just don’t recognize the rest.
With that, any editor that supports Language Servers can use those to provide better support to a language without implementing any details.
In this case, you can see some of the supported editors in the README itself: https://github.com/bash-lsp/bash-language-server#clients
The idea is also not new - python has had docstrings since the 90s. They never thought to show the full definition on hover or autocomplete, probably because it's noise most of the time, but a lot of people seem to like it.
LSP is _the_ way to write IDE integrations in the modern era.
(I maintain LSP tooling for F# and actively support at least half of the editors above)
There are also many different display/view methods that more sophisticated LSP implementations can implement. In my own LSP, we have different 'views' of the signatures/types/docs for
* hovers * signature completion * detailed type-level embedded docs in a separate pane
to name a few. It's entirely customizable.
If you're angry about it, feel free to produce something better.
These editors that have supposedly embraced LSP are perhaps throwing a bone to people who want to turn them into IDEs. If people don't use the LSPs, they keep the integrity of the editor - it is an editor, not an IDE. Whereas people who set up an LSP can go nuts. I'd like to see some stats about what % of neovim users are using an LSP plugin. I'm not.
You write a grammar in json and it gets compiled to a shared library via C.
Not sure what qualifies as doc strings, I discovered you have to use two hashes for multiple lines. More documentation on that might be helpful then I can adjust my comments to match its expectations. Also I'm pretty sure I must have broken it or something is going on in emacs lsp as it stops doing anything.
I especially like the vim like keybindings for visually selecting text blocks (although some things don't quite work the same).