I'd love to see more shell exploring things beyond POSIX. Text based stdin/stdout will always have its place, but having ways to express, serialize, and pass along data in more structured ways is quite nice.
Common worries I hear from people that were non-issues in practice:
- Not POSIX compatible: Nushell doesn't aim to replace POSIX, there's no problem with dropping back to bash to execute some shell snippets
- You need to re-learn everything: I'm not a huge fan of how the commands are organized, but I still didn't find it that difficult. nushell's prompt/readline comes with an inline help menu with fuzzy search. Hit CTRL+O to edit a command/pipeline in your IDE/editor of choice with LSP backed intellisense, type-checking and in-editor command docs/help. The syntax is very simple and intuitive.
- Just use python: Sure, but python comes with a lot of disadvantages. It's slow and uses dynamic typing. Static typing in nushell catch typos in pipelines & scripts before they execute. It also makes in-shell and IDE LSP tab-completions very accurate. Large files process quickly though it will still consume more memory if you aren't able to process all the data in a streaming fasion. It's like having jq but with autocomplete and it works on all command output & shell variables. Though if you really like python, check out Oil/OSH/YSH: https://oils.pub/
- All Unix commands output text, structured data is useless in a shell: `detect columns` (https://www.nushell.sh/commands/docs/detect_columns.html) - now it's structured. Or use `from <format>` if the command outputs CSV, JSON, INI, YAML, etc... Or don't, cause GNU tools work fine in nushell if you keep everything in text format
And there are other crazy features too.
- Write nushell plugins in your language of choice, plugins can work with structured data
- Plugins can run in the background and maintain state, nushell can automatically start a plugin when it is first used and stop the plugin when it is idle
- e.g. a plugin can open a SQL connection and use it across multiple commands. There's a built-in plugin for opening in-mem/on-disk SQLite databases
- Closures, generators, ranges, errors/exceptions + try-catch
- Ongoing work on DAP suppprt to allow debugging scripts from your IDE
- Create your own hooks to customize how different types of data are displayed. Display structured data in table/tree form, display binary data in hex, etc...
- Collect related commands/variables into modules. Load a module knowing that you can easily unload the whole module later, module contents don't pollute global state. Variable declarations, env vars and loaded modules are scoped to the current code block and disappear after the closing bracket, lowering the odds of a name collision.
- Native support of Polars dataframes to work with even moar data
- Complex parllelism: Message-passing/actor architecture background jobs. Turn-key parallelism: transform every element of a list in parallel - `par-each` (https://www.nushell.sh/commands/docs/par-each.html)
The biggest downside of nushell is that it hasn't hit 1.0 yet so commands occasionally get renamed. Expect that you may occasionally need to tweak a script to get it working again. Definitely a pain point.
Nowadays `job unfreeze` will do the trick.
But a lot of the structured data transformation use cases I encounter, I find myself tackling in DuckDB. It's a little harder for the simplest things, but it pulls ahead quickly. Or at least it does if you need to remember SQL anyway...
But I must agree that dedicated data analysis tools like DuckDB and jq are more powerful, intuitive, and performant. I guess what makes nushell appealing is how the data is already in nushell. It's where I stash any inputs I plan to use, any output commands produce and also any datasets I'm currently working on.
The true value of nushell is it's role as a data exchange that preserves typing+structure and in providing tools so ingesting structured data is easy and parsing unstructured data is not daunting. I'm less pushing for nushell specifically and more hoping that it encourages more people ro think about some larger questions:
- It's time to question the role of "UTF-8 text" as "the basic fundamental unit of data in the POSIX ecosystem"
- Typed/structured data brings significant value and is not harder to work with
- How can we improve data interchange between tools/apps without causing breakage? There's been quite a bit of thinking on if CLI tools can negotiate with each other to switch to communicating using higher-level data formats. Optimally it should work over common transports like SSH too. Unfortunately, I haven't seen any proposals that don't also introduce new problems. The nushell authors are looking into this as well.
- How can we evolve terminals from the simple, reliable text renderers (that they never were) into simple, reliable renderers of general structured data?
Related work:
- https://arcan-fe.com/ (obligatory mention: https://arcan-fe.com/2021/04/12/introducing-pipeworld/)
- https://github.com/nushell/nana (nushell GUI experiment)
- https://domterm.org/index.html
- https://acko.net/blog/on-termkit/ (defunct)
What do you mean?
YMMV, I’m mainly speaking from my own experience/history with powershell. For a long time it was the only way I knew of to manage various aspects of low level windows OS settings.
Using Windows is mostly GUI first, and traditional (DOS) command line second. Powershell is mostly used by admins.
Nushell on the other hand takes the (IMO more pragmatic) stance that the underlying app will most likely be writing strings to stdout and it's the shell's job to make it easy to discern the structure in those strings.
Perhaps a powershell wizard can show me that I'm wrong about this, but my feeling is that the powershell equivalent to this nushell is going to either call some external program (in addition to docker) or be quite messy:
$ docker ps | detect columns | where NAME == "foo"But merely concatenating processes which emit and consume data of known shape is something that the OS can do without an interactive shell. What you really need a shell for is those situations where a bit of duct tape and string is needed to keep the bits flowing... when "--format '{{json .}}'" doesn't work.
Re: ConvertFrom-Csv, touche, but I'm going to guess that it's not prepared to handle cases where the delimiter also appears in the output (spaces in this case).
$ docker ps | detect columns | select CREATED STATUS | get 1 # ConvertFrom-Csv equivalent
╭─────────┬───────╮
│ CREATED │ weeks │
│ STATUS │ 9 │
╰─────────┴───────╯
$ docker ps | detect columns --guess | select CREATED STATUS | get 1
╭─────────┬─────────────╮
│ CREATED │ 2 weeks ago │
│ STATUS │ Up 9 days │
╰─────────┴─────────────╯