Xonsh: Python-powered, cross-platform, Unix-gazing shell | Hacker News Reader

Xonsh: Python-powered, cross-platform, Unix-gazing shell

github.com

131 pointsthunderbong102 comments

It’s a cool shell. I used it for a half a year. And then it started crashing on me every ten minutes or so. That’s when I came to appreciate how important the stability of such a tool is.

I had the same timeline. Less instability, more that it had a few non-bash/zsh quirks so it was hard to adapt. Loved many things about it though.

There is known issue that is related to prompt-toolkit. You need to downgrade ptk version. Look here - https://github.com/xonsh/xonsh/issues/5241#issuecomment-1961...

The age-old question, do I strive to learn new tools that are better designed (Xonsh, Nushell, Fish)? Or old tools that are omnipresent (Bash/Posix)?

I like Xonsh, it’s pretty nice to work with, but it makes going back to Bash, when I have to, even more painful.

Take a look at https://github.com/xonsh/xonsh/issues before deciding to abandon the devil you know.

I prefer sticking with bash where necessary (where a script is the only thing that will reasonably work), and elsewhere using a programming language with testing, type checking, modularity, and compilation into something with zero or minimal runtime dependencies.

Better to live in paradise most of your life even if you have to go back to bash hell once in a while rather than endure that pain all the time

Seconded. I am very young, never bothered to learn bash, went straight for fish. ChatGPT is there to help if I ever encounter unintelligible “$@“&2>?!¿…

Gonna try nushell soon as that seems even more productive.

I also never bothered to learn Bash... `#!/bin/sh` all the way!

as such doing that yields different results based on am i on a mac, in debian, in a container, am i on a zebra-on-the-moon...

its all a bunch of loops and string manipulation in the end. in fact, awk can handle a whole lot of it too! and octave... and others, lol; so long as it does not turn into the python2 to python3 fun we've had in the past and we have some stability between versions; this is why i still choose bash (and the whole gnu binaries)

edit (formatting i hope, and python)

I'll admit that I haven't tried this on other platforms, but I was under the impression that the opposite should happen: `/bin/sh` loads a basic POSIX-compliant shell that should work on other platforms, with Bash, Zsh, and whatever else, which are extensions of a POSIX-compliant shell.

Blegh, now I have to do some digging...

sh will load whatever the OS decides it to open LOL; there is no rule. again why i just go with gnu bash as it is pretty feature rich and exec's other-cli-apps really well. the loops it can do, and arrays are just a bonus; if i need some more complex data structure, then there's another app for the job- that is not an interactive shell. just beware of the dragons when dealing with macos's old-ass bash; check out the version, go with new if you can

Nushell is great fun, but be prepared to encounter and fix errors when you paste code designed for sh/bash/zsh/fish. It's a much bigger step away from convention than the others.

I think the one that catches me off guard the most is:

    export FOO=bar
    BAZ=qux

Is now

    $env.FOO = "bar"
    BAZ = "qux"

It makes more sense the nu way, but old habits die hard.

I don't see much use for such pasting and fixing up— I generally find it more painful than other methods.

For short snippets, better to read, understand, and translate as you type.

For some shell integrations or environment setup tools, use fenv or babelfish.

For scripts of any length, just save it and run it. The shebang'll invoke bash or whatever is needed.

nushell requires double quotes around values? looks like powershell where everything-is-an-object of something-else

If you want those values to be strings, yes. For instance, it knows that this is a list of ints:

    $ [1, 2] | math max
      2

And here are some strings:

    $ ["a", "c"] | str join "b"
    abc

And there are tables

    $ [
       {name: "foo", value: 6},
       {name: "bar", value: 9}
      ] | where value > 7 | get name
    bar

And that's about as weird as the types get.

Nah, hell is everywhere. I'm better off using Bash when I need a basic script, then upgrading to Python if I need anything more substantial.

I recently (i.e., yesterday) migrated my 15+ year old bash config to zsh. zsh has some great quality of life improvements compared to bash and is basically 1-1 compatible. I had to spend about an hour migrating my prompt, but other than that it was a smooth transition.

zsh is now the default shell in macOS, so I'd say it's a safe bet if that's what you work with.

"basically 1-1 compatible"

careful, there are footguns in those words. It may seem like it's 1-1, but it's not. There are subtle differences especially in escapes. Bash uses \ escapes. Zsh uses % escapes. Zsh has builtin wildcard expansion. There are other differences as well but you can use the emulate command to emulate bash so it actually is 1-1.

Also, once you've made the switch to zsh - checkout oh-my-zsh (https://ohmyz.sh/)

it's pretty funny that Zsh only came out one year after Bash... in 1990

I switched a few years ago and while there's a lot to like about (and power in) zsh, there's a lot i really dislike about it. For starters, it adds so much additional functionality and compatibility that the documentation (man pages are terrible). Also, the additional history & variable expansion capabilities are messy/ugly in shell syntax (imho). I think ultimately the problem is that, for shell scripting, bash has clearly won, but other shells show that there's room for an alternative specifically for a user's interactive interface...but zsh didn't get the memo so is trying to be all things to all people. One of these days, i'll probably swing over to fish...if I can get the energy to change my environment yet again.

Personally I'd prefer to have to learn no shells at all, but since that's not possible I'll stick with the one that's most commonly installed, which is bash. Similar feeling with an editor - vanilla vi instead of learning and depending on a slew of fragile extensions that only work on a personal laptop. If the challengers supplant the defaults on servers in these domains at some point, I'll learn them then.

I feel like having ipython available whenever I feel like it from my shell is the best of both worlds

you can use both.

whenever I have some task to deal with data manipulation, i.e. fetch a json, map/filter/reduce on it, save it as some format, I reach for nushell.

If it's just process management or day to day trigger of command in a folder, I use zsh.

Learn the new tools. I’m working on my own machine 99% of the time, and if I’m on a remote machine, there’s a 90% chance I’m running something automated there. I’m not going to handcuff myself to the baseline for the sake of that .1%.

There’s no way I’d go back from Fish to Zsh or Bash on my daily driver. It’s just too pleasant to give up just because of “what if?”.

If it's Python, is it really a new tool?

The big problem is that bash is more or less portable (almost everything has bash in the box). They need to start convincing distros to include and/or default xonsh, to really make it worthwhile.

In my view, yes. It would allow you to extend you shell easily, for example do something with the history, like setting a blacklist for commands that should not appear in the history when using the up key, but still retaining the execution of such a command in the history for poor man's auditing.

I use zsh on my work and personal computers. I'm not ssh'ing into boxen these days. But when I do, I'm not doing anything more than reading logs to figure out why the userdata on an EC2 didn't work as expected.

I try to use posix standards when convenient, but I'll switch to bash at the first sign of posix complexity.

Xonsh seems like I'd have to type a lot more than I do with zsh. I would also be concerned about not being able to give my team members the same command I used without forcing them into a non-standard shell.

I don't use fish because I've only met one other person IRL who used it. Everyone I've worked with has use bash, zsh, or ksh (I'm glad I left the ksh company before they had to rewrite all those ksh scripts).

Also, Bash is staying for now. posix will most likely always work for the foreseeable future. Zsh seems to be the new Bash, but I have yet to see anyone put zsh in a shebang at work.

I found xonsh to be configurable to be extremely similar to zsh. I don’t see that it’s «more typing».

> I have yet to see anyone put zsh in a shebang at work.

I put it in shebangs for macOS scripts nowadays, since it's been the default shell on macOS for a little while. That's a niche for sure, but still.

If you're new and still trying to get into the industry, try all the new tools. Drive them hard and try to break them, so that you can find bug fixes you can contribute. Just go nuts, and let yourself steadily build up a backlog of unique, public, referencable commits you can show employers.

Once you're already established and comfortable, it's up to you if you want to keep trying the new flavor of the week. People gravitate towards novelty at different parts of the stack: Some people love running FreeBSD or Alpine, but stick to Bash on top of those; others, like me, try to stick with Ubuntu whenever possible, and mess around with things like shells and tiling window managers. Others even return to Windowsland and instead focus all of their efforts on innovating at the highest levels of what they can do with C# and actually making money with an innovative business model.

But you'll never learn where you don't enjoy the thrill of seeing something new break on you if you don't have that initial "question everything" phase.

Just go for it. I've been using Fish as a daily driver for something like 14 or 15 years now. I have no regrets about it.

If I want it on a server I'm using, I (*gasp!*) just install it.

(I still write Bash sometimes and that's not really a problem, either.)

I don't think Fish counts as "new" at this point, given that it's been around for almost 20 years now. "Boutique", perhaps?

I've been using xonsh since 2018 - on both Windows and Linux. I love it. It's not as responsive as a C based shell (e.g. Bash), but you get over it and the pros heavily outweigh the cons.

If you love Bash scripting, xonsh isn't for you. For everyone else, xonsh is.

The only serious headache I have with it - very poor compatibility with fzf. None of the main developers use fzf so they don't feel the pain. As an example, doing:

    git checkout $(git branch | fzf)

doesn't work because the result of the fzf command has a newline. So you need something like @$(...).

What I really want is to swap the default TAB autocompletion with an fzf selection, and there's no easy way to do it. You have to get deep into the underlying prompt_toolkit library and do a lot of surgery. It's doable, but I haven't had the time.

I've been using Xonsh as my main shell for about the same time. I love it as well.

However, I feel like your criticism re fzf is not really fair, because I run into this with other tools quite often. So often in fact that it took me only a second to convert your command in my head to this:

  git checkout @($(git branch |fzf).strip())

As I said, it's doable, but painful. If I show your command to most non-Xonsh users, it would be an immediate turnoff. Do they really have to put strip at the end of each command? Or keep track of which need it and which don't? It's very unergonomic.

And the history of fzf support in xonsh has been much worse. Until recently, the command you wrote was not guaranteed to work, because doing anything with fzf under $() would (seemingly) randomly not capture the results of fzf (it would often be any empty string). This was a long standing issue for years, until we finally had a user who knew the xonsh internals well enough to go and debug it.

So people like me had to find alternative ways to use fzf (e.g. attach it to a custom keybinding for specific tasks). I had to modify xontrib-fzf-widgets (because that itself failed often).

Take a look at these issues:

https://github.com/xonsh/xonsh/issues/5190

https://github.com/xonsh/xonsh/issues/5189

https://github.com/xonsh/xonsh/issues/2404 (note that it took 4 years to fix this - and until then your command would not reliably work).

https://github.com/xonsh/xonsh/issues/3548

https://github.com/xonsh/xonsh/issues/3035 (probably a clone of 2404).

The reality is: If people want to use fzf in xonsh the way they do in bash/zsh, they will be in for a lot of pain.

This bring back memory of my younger self playing with the BASIC Interpreter cartridge in my old Atari 800.

With this setup, you can...

* create a BASIC program right at the command prompt

* execute instructions to read and write to mass storage.

* debug your BASIC program at the command prompt

* direct access to memory addresses via peek and poke commands

> xonsh is Bash-wards compatible in the ways that matter, such as for running commands, reading in the Bash environment, and utilizing Bash tab completion

What does in the ways that matter mean? It would matter to me that it is 100% Bash compatible, else I won't use another shell in addition to `zsh`.

It's not POSIX compliant.

It's Bash compatible in many/most ways, but there are always exceptions. If you want a Bash-like experience in Windows, xonsh is a good fit.

"Unix-gazing", "Bash-wards", I don't even understand what it is yet.

I wish there was a comparison table with xonsh, fish and nushell. Maybe also bash and zsh as baseline.

The rc shell [2] from plan9 seems more simpler (and saner) than bash. At least you can read the mapage of it[1].

[1] https://www.mankier.com/1/rc

[2] https://github.com/rakitzis/rc

why not just run interactive python when you need it?

A shell should be interoperable between users. If everyone on your team uses xonsh, then great, by all means. Otherwise, you're in your own little esoteric world.

a shell has to be stable, a python environment is not. There is just too many ways a python package can break. If python had a good package system perhaps it'd bring something to the table.. but python is by far the winner of the programming language with the worst packaging ecosystem i've seen and used.

You can use your own tools in addition to those necessary for team stuff. In fact, I'd recommend it. There's no point using the same tools and rigour that you would with a team. Use a different version control system. Use a different shell. Have fun with your computer!

> why not just run interactive python when you need it?

Because writing shell scripts in Python/xonsh is much more pleasant than in Bash.

You can run the bash shell interactively and write python scripts

why limit yourself to just one. I'm just talking about the interactive shell here..

> why limit yourself to just one?

That's precisely the rationale behind xonsh! You're not limited to Python. You're not limited to Bash. For the most part, you get the best of both worlds. And you can still run Bash scripts in xonsh.

an interactive shell really _doesn't_ need to be interoperable between users in the first place, but even a scripting shell doesn't need this either.

This is what the shebang in your scripts is for. If you need to write a script that is going to get shared, you have (at least) two perfectly good options:

  - use bash (or some lowest common denominator)
  - use a fancier (better?) shell, like fish/xonsh/nushell/oil and just require that everyone have that shell installed (they don't have to use it as a daily driver).

xonsh is not really any different from other shells - it can be installed as an isolated environment with its own runtime (in this case Python)

A third and usually better option is to use a dedicated scripting language - this is of course an opinion, but many people have recognized that anything longer than ~50 lines and that will need to be iterated upon in the future is a bad fit for a shell-like language.

Do those more object oriented shells expose more substitution and streaming goodies? For example do they have process substitution - i.e. <(foo | bar)? Or do they use memfd instead of temporary files on Linux? Do they have first class fd, processes and pipelines manipulation?

For example the thing that turned me completely off PowerShell is that its pipelines are sequential not parallel. I use shell a lot, because it has streaming built-in. Many (also standard) libraries do not work around streaming, which I find is the reason why they often crumble when there is more data. For example Python's ZipFile or TarFile work around listing members should have been iterators.

I'm a DevOps engineer and I agree with the mission of Xonsh. The shell is broken.

Make sucks, it is a c build tool we have Frankenstein'd into a task runner.

Bash is missing a huge amount of features of modern programming languages, and has a bunch of foot-guns baked in by default ( not setting -e, etc. )

I would love to see a modern language that:

- has a similar mental model a d syntax to other programming languages

- built in argument parsing, declaration, and help string generation

- designed for "scripting" - calling out to other binaries and gluing tools together

There are a few:

- Elvish: https://elv.sh

- Murex: https://murex.rocks

- Nushell: https://www.nushell.sh

Each have their own strengths and weaknesses

Python has issues with dependency management. I don’t know that having your shell be based on Python is such a great idea as you could easily get yourself to weird situations.

That's a weird take given bash doesn't have _any_ dependency management. At least with XonSH you get something? Also, I'm not sure what issues you're hinting at - my experience with python dep management is that, although the tooling is terrible, the core import system works fine.

> That's a weird take given bash doesn't have _any_ dependency management.

I guess that is the point here. With bash there won't be installation or maintenance-problems because of some strange dependencies or unexpected changes.

Though, xonsh seems to only have 3 dependencies, of which one is for unit tests, one for syntax highlighting and something for the prompt. So I guess the danger is kinda low to run into problems.

No errors printed means no errors exist. Unrelated, why won't my script work?

What?

To move from the abstract to the concrete: I've been using it for almost 6 years and this has not been a problem.

OK. I lied. Since I installed xonsh and its dependencies in my user directory and not system wide, every time I upgrade my Python to a new major version I have to reinstall xonsh. Usually just one command will do it.

And it's not even in a virtualenv.

What I really want is all the shell commands to have an object (e.g. json) output mode. Then all my "glue" between tools becomes less bespoke and more standardized, e.g. jq. Instead of analyzing output to build weird grep and awk statements, I would just need to remember a couple of jq patterns, take a quick look at the output, and use one of my patterns to extract the data to stream to the next command.

Sounds nice. But I think we all know how that would go (insert xkcd on competing standards)

Nushell?

https://github.com/nushell/nushell

I want it to work in all the shells, which is why I want the functionality inside the tools themselves.

Sending you thoughts and prayers, then. Although, I am not very confident it's within God's capabilities.

I have a recurring nightmare where web developers create a one world government where they enforce a law requiring all data be converted to JSON before transmission. Existing network protocols are replaced with JSON-compatible versions, and specialized JSON parsing ASICs become a necessity in network devices. The nightmare ends with the collapse of civilization.

Didn't that happen with XML and web services? We survived that one.

https://github.com/kellyjonbrazil/jc - "CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts."

Thank you for the link. It still would be great built in functionality for the tools themselves.

Take a look at Marcel, https://marceltheshell.org.

It passes Python values instead of json, but otherwise works as you described.

I use xonsh. Doing simple math on the command line reminds me of my QNX days back in the 80's. And being able to do python on the commandline is awesome.

Also, the rc file is way easier to read and manage.

I don't know why they advertise as being like bash. It's absolutely not like bash except that their default prompts look similar on the command line. The similarities pretty much end there.

Xonsh by itself is pretty cool and I hope it continues to grow.

However, I feel like Xonsh needs a killer app to really demonstrate the advantages of the semantics Xonsh provides. Maybe something like Rawdog (https://github.com/AbanteAI/rawdog) + Xonsh?

Interesting. I've been mulling around in my head a shell which embeds a Lisp. Well aware the idea isn't particularly ground breaking but I do love the idea of better languages embedded right within the shell, but not quite a REPL for that language.

>> I've been mulling around in my head a shell which embeds a Lisp.

There are several projects with the same idea: https://www.cliki.net/Lisp%20as%20a%20shell

Oh, I know they exist already.

Emacs has eshell which is just that!

I use xonsh as a shell for few years but I think I will go back to another shell as I am too frustrated with incombatibilities. That's being said, my xonsh scripts are there to stay, I still think that's an excellent way to build scripts.

Used to use this! It's a great shell but I'd have to be using the CLI like 20 times more than I do to justify it.

Using the same tool everywhere is nice because... that way I don't forget how to use BASH when I eventually need it.

I tried it a while ago and encountered problems with ncurses-based applications not rendering right. I ended up going back to zsh and then later fish.

I believe they solved that problem and made it the default years ago. And there was a fix for it as far back as 2018, when I started using xonsh.

I hate to say it, but I've completely stopped using Xonsh (which I adore) since ChatGPT-4 (whom I also adore) came out. Xonsh lowered the friction and cognitive load to get some jobs done in an elegant or at least sufficiently rapid way for me, and I am grateful. I can still see LLMs, many of which are benefactors of significant RLHF of the two parent languages, making great use of such a tool - especially paired with a nix-style approach to handling dependencies and environments.

I haven't tried ChatGPT-4, but all of the local models I have tried have been garbage at writing Nix. I only recently started playing with them, so I'm sure there are improvements I can make, figuring out langchain to import documentation seems promising. I'm hoping some Nix Nerd with enough compute will train a model specialized for Nix, but I know that is unlikely, however.

"I need to build a skyscraper. I haven't tried using steel, but using wet spaghetti has been garbage."

The vast majority of humans on the planet haven't been able to afford to work with ChatGPT-4 (and I don't consider M$'s offerings of it to have nearly the same range of use). Perhaps there are good reasons our interlocutor hasn't tried ChatGPT-4. I also admire that someone has been testing with local models; we benefit from their experience as well.

True, but we're not talking about "the vast majority of humans on the planet", we're talking about the sort of person who plays around with Nix - who can probably afford gpt4's $0.03 / 1K tokens.

But I absolutely agree with your 2nd point!

I understand that claim. I'll agree the vast majority of Nix users I've encountered could likely afford to use ChatGPT-4 if they wanted.

As to our domain of discourse, I'm here to speak about what we owe to all people, and I'm not here to make that assumption about our interlocutor. Maybe you know our interlocutor, but I don't think your standalone quote is a morally appropriate argument with a stranger. Setting aside those who might be the sort to possibly play with nix who haven't, I've also met people who have played with nix but cannot afford ChatGPT-4, or don't have access to pay for it. There really may be good reasons our interlocutor hasn't tried ChatGPT-4, and I think your joke can be easily misinterpreted as a flippant response or a putdown.

I appreciate the meme though. You're correct about the vast difference in ability between ChatGPT-4 and every other model the public can access.

You keep saying “chatgpt-4”; I wonder if you’re aware that you don’t need to pay $20 a month (which is what that costs) to use gpt4. If you use it through the API instead, even a fairly heavy user comes out to a dollar or two a month.

I understand one can pay to use APIs to various models offered by OpenAI. API use with maximum context does add up for significant users. It does cost something, and not everyone can pay that cost.

Even GPT-4 has been hit or miss for me. I think it's less about the model and more about the data. I'm intentionally aiming to write my home-manager stuff as modern as possible (e.g. flakes, recent home-manager syntax) but most of the content out there uses older paradigms, so it's often hard or impossible to get chatGPT to produce quite what I want.

Still helpful, just not as helpful as it is for like, python 3.11.

I probably get more mileage out of it because I stick to one giant skiddieable configuration.nix file (~2k lines so far) for a base on a single machine, with some scripting to get some stuff done after that. I don't use so much of what makes Nix radically powerful, `/nod`. Like, I'm runnin' a noob's GNU/Systemd/Loonix up on my Rube-Goldberg Machine for the 'tism points, my sibling in Christ; it's spaghetti, and I wish I could share it with Saint Terry. So, yeah, I can see how for those seeking modern nix specific affordances, it's a much worse experience. =/. One can hope that the next model will make it rain.

There are so many facets of the nix ecosystem that have so little data in the training corpus, and hopefully that will be fleshed out further in future generations. Probably no way around RLHF from some experts, and whatever magic sauce they can cook up after that either, to get the right sort of data.

Yeah, that would be dope. You're also right that nothing is ready for writing Nix well, though ChatGPT-4 can do some work in basic configuration (3.5 really can't). It may also be worth considering how a surprising amount of NixOS content and infrastructure is really bash-driven. Perhaps nixlang is a metalanguage computing over quite a bit of bash object-language tooling. I think ChatGPT does have something to collaboratively contribute to many aspects of using nix, but I do think most of the assistance is limitingly generic. I still think the pairing is valuable even if ChatGPT doesn't work on any of the nix portions of it, though I'm not convinced it couldn't be prompted not to construct or at least assist in constructing with supervision many worthy Xonsh-related nix environments.

Love the Python syntax so I’m tempted to try this, but I wish it incorporated some of the ideas in nushell (e.g. structured data).

Check out marcel (https://marceltheshell.org). It's yet another pipe-objects-instead-of-strings shell (like nushell). Unlike nushell, you pipe Python values. Marcel has no sublanguages (like awk, sed, ...). Instead, when logic is needed, you write Python code, delimited by parens. So:

    (USER)

prints the value of the USER environment variable.

    (f'Hello {USER}')

evaluates the expression inside the parens and prints "Hello jao" (for me).

    ls | select (f: time.time() - f.mtime < days(2))

ls finds files in the current directory, pipes them to select, which keeps the files that were modified in the last two days (days is a builtin function).

Lots of examples on the website.

Can I use marcel on Windows? If not, any plans on supporting it in the future? And even better, if I could install it with:

    winget install marcel

No, it does not run on Windows. I'd love someone to help port it to Windows ...

I wonder how does the `import` command work.

Does it first check if `imagemagick` is installed first and then performs a screenshot?

I wonder if this could be any good as a bash replacement in GitHub actions

Obligatory mention of scsh (https://scsh.net/about/what.html) and somewhat Tcl (https://www.tcl.tk/man/tcl8.5/tutorial/Tcl26.html, it'd benefit from some DSL work to abstract <@ and >@).

Obligatory mention of rash[0] which I absolutely adore but because it never got the love it desires, the REPL experience is quite lacking. The scripting however is genuinely pretty delightful

[0] https://rash-lang.org/

whats the advantage to using this over interpretted python?

You can mix shell commands and python.

wtf does "unix gazing" mean?

GAZING, it's like gassing but said with an easter european slang, try to read the word and say it slowly "gazing". I am "gazing".

This reply doesn't help me at all

Read it as "with an eye towards Unix", i.e. targetted at Unix users.

Actually, I'd say it's targetting Unix users who have to use Windows :-) I initially installed xonsh on my work laptop so I could have a bash-like shell in Windows.

Reply on news.ycombinator.com