Show HN: A smarter Unix shell and scripting environment
github.com
github.com
no no no you're missing the point: on POSIX systems, go with XDG by default. cmon guys it's 2023 not 2003
You can override that behaviour if you want to use XDG but making it the default could break backwards compatibility, result in confusing documentation for non-POSIX users and introduce a lot of additional development and testing just to fix something that already works fine.
Like with any open source project, if someone else is willing to commit some time into solving these concerns then I'll gladly merge it. But I need to be pragmatic with how I prioritise my development time.
If it offers substantial benefits over Fish, then I'd gladly take a look.
As it stands, the description is quite brief and doesn't talk as much about other shells and feature comparison.
Improving this side of things is something that's constantly on my mind though. Plus if anyone else also wanted to contribute then I'd welcome that with open arms.
While I'm spending quite some time in my shell and the extend that its a core part of my system is quite considerable and makes switching to another shell altogether a huge time investment.
Nonetheless, I'll always keep an eye on promising alternatives and eventually switch over once the functionality missing gets apparent enough on a constant basis as it happened for me when switching from fish to zsh.
As someone that went the other way ~5 years ago, I’m curious what made you switch :). Are you btw on a hand-rolled config or one of the distributions?
(For me, it was that I was maintaining a ~400 line zsh config, and then learned that fish did most of the things I had configured “out of the box”. I considered switching back for increased POSIX compatibility since I often write bash scripts at work, but since Fish 3 added POSIX-compatible syntax for most pain points, I haven’t kept up with zsh.)
> open https://api.github.com/repos/lmorg/murex/issues
As a Mac user, I'd expect that to launch a web browser, but it appears to be downloading and parsing the JSON?
echo 'Something or other' | sed 's/or/and/'
and echo 'Something or other' \
| sed 's/or/and/'You can just end each partial line of such long commands with a pipe sign and keep writing the rest of the overall command on subsequent lines, except for the last partial line, where you just press Enter to complete and run it.
This naming conflict is actually one of my regrets within Murex. I was a couple of years into development on this shell before I obtained a mac test system. I think most of the Murex users in the early days were also Linux users too. So I wasn't aware of the conflict until there was already too much code written in Murex to be worth the risk renaming the builtin.
However can mitigate this by running `exec open ...` -- you can set that as an alias as well, eg
alias o=exec open » file ${which open}
/usr/bin/open: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/open (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/open (for architecture arm64e): Mach-O 64-bit executable arm64e
Whereas on Linux it is a shell alias. Doesn't appear to be universal either (doesn't exist on Arch, for example). It was a recent change too, added to Debian 11 (released 2021), which is years after it was added to Murex.As an aside, I wouldn't be at all surprised if the `open` alias was added to Debian to copy (and I mean this affectionately) macOS. Otherwise `xdg-open` might have just been called `open` from the beginning.
There is the same command function wise on Debian Linux. Alias or not makes no difference.
Debian Linux is the base of almost all Desktop Linux boxes out there. (There are of course niche distris, but the overwhelming majority is Debian based; I count Ubuntu as Debian based.)
There is nothing "specific" to my shell environment. Having the `open` alias available the default. I would need to change things manually to diverge form this default. That would be "specific to my shell environment", not the other way around.
Historically `open` comes form NextStep, so actually Apple copied it.
On Linux it's usually a symlink as there are other implementations then the one in `xdg-utils`, and there have been also other programs with that name in the past.
First of all you describe it as an alias, which is very different to a symlink. A symlink is a file so will be available system wide. Whereas an alias is only available to shells that have had that alias added to (and it's pretty common for embedded shells, like in IDEs, to not pick up default aliases). So my points were about aliases not symlinks.
For what it's worth "open" doesn't exist on my Ubuntu servers either. And it was added to Debian only 2 years ago. Murex has been around for nearly a decade. So I can hardly be blamed for adding a builtin with the same name as something that hadn't yet been added to Linux.
> Historically `open` comes form NextStep, so actually Apple copied it.
Apple didn't copy NextStep, they bought it and ported/rebranded a lot it's tech. Also I wasn't claiming Apple invented the concept of "open". I was saying I suspect Debian got the idea from macOS.
---
Anyway, this is all moot because the `open` builtin (Murex) already checks if `open` is a TTY (ie you're not piping the output of `open`). You can define how commands get opened when it's a TTY as well https://www.murex.rocks/docs/commands/openagent.html -- this is how Murex can inline images into the terminal -- so i can just add a "if no open agent is defined fallback to exec open" type condition. Then everyone is happy.
As to why `open` builtin reinvents `xdg-open` -- well that's because Murex needed something cross platform and there wasn't a reliable way to do that at the time. So I had to roll my own. Now the landscape has changed somewhat, I daisy chain the builtin to call the system `open` command on supporting platforms.
Maybe. But I just don't know why this discussion started in the first place. There is a `open` command on likely almost every desktop Linux box. So it's not Mac specific (which was my sole point in my first comment in this thread).
> Right, you said "alias" multiple times earlier. Symlinks and aliases are not the same thing.
You're technically correct. My fault!
I've called it alias as it's a symlink coming form the Debian alternatives facility. The alternatives mechanisms manages system wide command "aliases"… I should have been more exact in this point.
> I was saying I suspect Debian got the idea from macOS.
I have no prove but I doubt that. There were other tools on Linux called `open` before MacOS X came along as far as I know. The whole thing seems to be rooted in BSD (where NextStep got it's Unix parts form).
> For what it's worth "open" doesn't exist on my Ubuntu servers either.
Makes perfect sense. It's a desktop tool.
> And it was added to Debian only 2 years ago.
I don't think so. I had `xdg-utils` and it's `open` command installed for many years. It's at least 17 years old:
https://cgit.freedesktop.org/xdg/xdg-utils/refs/tags
> Anyway, this is all moot because there are easy workarounds for the naming conflict
I agree. The Debian alternatives system was made exactly to handle this kind of name conflicts around commands. So there's no big issue, indeed!
I'm running Debian Testing so I had this command in fact since many years. Didn't know that it's actually quite "new".
Now I can understand how it could happen that Murex created a command name conflict. External tools don't test against Debian Testing most of the time…
xdg-open was called xdg-open because the name open was already taken by another Linux command - to run a program on a new virtual console. However, since virtual consoles are used far less than they once were, eventually people decided to reserve the obvious name for the common function - so open got renamed to openvt, and open became an alias for xdg-open. It also helped prevent the confusion for people coming from macOS to Linux, trying to use open and getting a completely different command instead, and then asking “why do I have to put xdg- first???”
> As a Mac user, I'd expect that to launch a web browser
As a Linux user I would expect the exact same thing.
If I'd need to install another tool to take my scripts with me, I'll probably would just use another scripting language like Python.
Whereas Murex colours pipelines with type annotations so that it can perform it's magic with your existing system's coreutils.
That's not to say that Nushell's approach is wrong. It's a popular approach used by a few alt shells, including PowerShell. But it does come with trade offs:
1. your coreutil muscle memory needs to remember whether you're in Nushell or Zsh (eg `ls` flags will differ depending on your shell)
2. the developers of Nushell need to re-implement coreutils to expose the genius of their shell. And as we've seen on here before, developing and maintaining coreutils is far from a trivial task
3. any other CLI tools that people might use that might not be common might also not support Nushells magic
These problems doesn't exist for Murex because it is basically an abstraction on top of POSIX.
To be clear though, I'm not making a criticism about Nushell. Murex has it's own problems and trade offs as well. So please take this comment only as a description of where our approaches have differed (I think it is great that there are numerous options these days).
This just sounds like a non-starter for me. I'm already irritated by having to deal with both BSD and AT&T flavors.
But, to each their own.
https://murex.rocks/docs/tour.html#filesystem-wildcards-glob...
But I do think being able to work with your systems coreutils is a valuable convenience too.
> the developers of Nushell need to re-implement coreutils to expose the genius of their shell.
Yes, or if Nushell is going to try not to be interoperable with existing POSIX commands, then the project needs to focus much more on perfecting the design of its exposed library of builtins.
It seems to me like tools like bash, lmorg, nushell, and PowerShell exist in some kind of continuum (which I think is, from left to right as I've indicated them above, the amount of "smarts"/type information objects returned from commands have). Perhaps not coincidentally, the further right you go, the more what you have starts looking like a programming environment than a shell to execute other programs.
I have written a bunch of scripts that basically manages my whole home server using podman by leveraging `podman --output=json`. Most new tools support json output that make it easy, and by adding an alias that adds the `--output=json | from json` (or the equivalent for each command) it works pretty much the way you would expect.
For others that don't, you just need to add a little parsing to kick it off. Here are some examples on their github[0]. Once you have what works, just add it as an internal command, and it's a "fixed" problem.
I personally prefer nushell's method, as it allowed me to add tools that do some more advanced stuff pretty quickly.
I don't have Windows to test Windows `netcat` on but the Murex code would look something like this:
netstat -ao -p tcp -b | [Proto..]r | tabulate --separator " +" | [Proto "Local Address" "Foreign Address" State PID]
# [Proto..]r -> select every line after the regexp expression "Proto"
# tabulate --separator " +" -> by default columns are split on whitespace. But here we are saying use two or more spaces
# [Proto "Local Address" "Foreign Address" State PID] -> selects columns (we already have column titles from `netstat` so why reinvent the wheel?)
Likewise with your `podman` examples, in murex this would look like: function podman {
cast json
exec podman --output=json @PARAMS
}
You can even configure Murex REPL to only autocomplete commands that support JSON input from `podman`: method define podman %{ Stdout: json }
...so now when you type `podman | <tab>` you only see commands that are compatible with JSON.I do have a lot of respect for Nushell but I've been using Murex as my primary shell for longer than Nushell has been around so a lot of these edge cases have been solved in Murex too. I just don't do a particularly great job at advertising it :)
I do script with bash, however.
find . -name 'somefile'
should have been just find 'somefile'
Ie, defaulting to the current dir and default to find via name.find | grep somefile
because I can also use all the pattern syntax I use elsewhere in grep commands.
One of the reasons I use zsh is because of all the plugins that exist for it. With them I have the feeling that they boost my productivity. But the real reason is, zsh feels more like it does what I want/expect compared to bash (, that is probably subjective?).
Not that I am a personal Powershell fan (in-memory commands and ugly commands is the ugly part for me), but "next-gen" shells with typed streams are not exactly the novelty.
If people are already using and like Powershell then I'd recommend they stick with Powershell. But if anyone is using Bash, Zsh or Fish (for example) but longed for something a little more intuitive (fewer footguns, smarter handling of structural data, etc) then Murex is a good option.
It's also worth noting that I've modelled the interactive shell on IDEs rather than readline. So the UX in Murex is, in my biased opinion, one of the best out there. However I'm always open to feedback on anything that can be improved in that department. Or in fact, anything regarding the shell.
But I get you. PowerShell is intended to harmonize server management and yours is a general purpose shell. Sounds like a very reasonable differentiation.
Do you write articles about how you make coreutils output structured? Is that a command specific adapter or how does that work?
POSIX does standardise some commands[0], albeit not many and good few of them are implemented as builtins even in Bash. But it also defines STDOUT (etc) files. The way Powershell passes data is as .NET objects. The way Murex passes data is as POSIX byte streams but with type annotations passed out of band. This creates some extra overhead in serialising data but the advantage is that it works with all of your existing tooling without modification.
> Do you write articles about how you make coreutils output structured? Is that a command specific adapter or how does that work?
I haven't but that's a good idea.
The way it works is just that you add type annotations. The default type is assumed text data, potentially in columns, like you'd see from `ps aux` or `ls -l`. There are also some builtins (eg [1][2]) that are data type aware.
So if you just use the most basic features of Murex, you get a few footguns removed (eg by default variables containing spaces don't get split into multiple parameters) and a slightly different syntax to Bash but all the same CLI tools work. However if you want to invest more into Murex then you get greater power from it eg the ability to inline SQL queries[3], write unit tests[4], better error handling[5][6], etc as well as being able to do `jq` like expressions against a multitude of object and table formats.
I really do need to get better at publishing it's features though!
[0] https://en.wikipedia.org/wiki/File:POSIX_Utilities.pdf
[1] https://murex.rocks/docs/commands/index.html & https://github.com/lmorg/murex/blob/master/examples/table-in...
[2] https://murex.rocks/docs/commands/foreach.html
[3] https://github.com/lmorg/murex/blob/master/examples/inline-s...
[4] https://github.com/lmorg/murex/blob/master/config/defaults/p...
[5] https://murex.rocks/docs/commands/runmode.html
[6] https://github.com/lmorg/murex/blob/master/examples/try-catc...
One thing I enjoy about fish is that it's not just a shell, but also a modern scripting language alternative to sh/bash. Seems like I would lose this with Murex?
(There's no right or wrong choice for Murex here, just that has to choose one way or another.)
So there is a little more to re-learn than you would with Fish but it shouldn't feel alien to read and write.
To me, bash-completion is horrible. I am looking for a tarball to unwind, and I only remember that it it started with "foo-" and it sure did not end in ".tar" (trying to save keystrokes, y'know). Trying C-i does not pull up anything. Argghh. I am on a machine with bash-completion!
---
I often hit C-a then "a" to get an invalid command (still saved to my hist), or C-a C-k if I think there is nothing else going into the kill-ring. bash-completion absolutely messes with me.
If it works for You, that is great. I really dislike it.
fzf.bash just turns `kill` into `ps (args) | fzf`, and does not yet have an equivalent for `killall`.
I have to `kill -9` neovim once or twice per week due to <https://neovim.discourse.group/t/how-to-debug-intermittent-n...>, that's probably the most frequent culprit.
I really need to compile a debug version and help get that sorted.
zstyle ':completion:*:*:kill:*:processes' command 'ps xo pid,user:10,cmd'
I also filter some processes that I almost never want to kill to clean it a bit up: zstyle ':completion:*:*:kill:*:processes' command 'ps xo pid,user:10,cmd | grep -Ev "(ps xo|firefox|/bin/zsh|-zsh|runsv)"'
As with most things in zsh, you can make it as fancy as you want, but this is enough for me.zsh is really powerful, but the downside is that a lot of these things have a bit of a "magic incantation factor" unless you're pretty familiar with zsh scripting. I guess this is why people use oh-my-zsh and things like that.
All things considered, I'd rather optimize for the 99% of the time I'm on my local machine than the 1% I'm on a remote machine.
https://github.com/lmorg/murex/blob/master/examples/inline-s...
Doesn't seem to be valid quoting?
https://github.com/lmorg/murex/blob/master/docs/tour.md#quot...
HN FAQ: