Hacker News Reader

Comment by pmarreck | Hacker News Reader

I've been on a longish search for a bash replacement (or at least "extra substitute") in the realm of "more complex commandline scripting" (functions, utility scripts, etc.). Naturally, every option is a set of different compromises.

I think I've finally settled on Lua- It's small, coherent, surprisingly fast (especially with LuaJIT) and has the "virtually no startup cost" I was looking for (which ruled out, for example, Elixir for a lot of things).

And, I've finally figured out how to bang my nix-darwin flake.nix config in a way that gets it AND a bunch of popular libraries installed together and all seeing each other.

Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.

View on HN

GNU Guile [0] is pretty cool.

[0] https://www.gnu.org/software/guile

Can you expand on using Guile for shell scripting?

I was aware of it, but not familiar until I looked at the link you shared. Looks like a Scheme, which for the uninitiated is a different approach for shell scripts.

Yes, there are some basic examples here [0] – otherwise it's just shebang and you're good to go; it can be a rabbit hole where you start liking it too much than what you really should (you wanted simple scripts, now you are finishing scheme book).

It's consistent, "infinitely" terse (you can create dsl-like apis for your problem domains), proper programming language.

First time I used it for scripting was when I had to process some xmls and do bunch of processing and gluing together – I was impressed how quick and easy it was without knowing anything about it and how readable the whole thing ended up being.

[0] https://www.gnu.org/software/guile/manual/html_node/Scriptin...

Is Guile really that good at the shell scripting "thing"? Like, the bash superpower is that it makes it extremely easy to run subprocesses in incredibly flexible ways (in the background, foreground, in pipelines, with stdout/stderr redirected to files, etc. etc.). Can Guile do that smoothly? I generally find that doing things like this is way more awkward in most "real" programming languages compared to shell scripting languages.

I know I’m in my own little corner, but stringing process together is the shell/unix superpower.

The “Unix way” stands on the shoulders of stdin, stdout, pipe, and fork.

And the shells make that a first class concept. Breaking up the command line and linking processes together is the #1 job of a shell, everything else is gravy.

The scheme and lisp shell takes fail here, just because you have to go through the gymnastics of syntax.

My fantasy would be to have something like this:

  (! ls | (myfunc) | sort -r -n | >list)
  (“100 x.txt” “200 y.txt”)

Piping system commands into my own functions within the environment.

Sure, I could do:

  ls | myfunc.scm | sort -r -n

But then I can do that with anything, not just scheme (there’s that Unix superpower again).

You also have to go through gymnastics of syntax to write a shell program that anyone could reliably use on any data, with any file names.

  $ txr
  This is the TXR Lisp interactive listener of TXR 299.
  Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
  Poke a few holes in TXR with a fork before heating in the microwave.
  1> (flow (glob "/proc/*") (take 10))
  ("/proc/1" "/proc/10" "/proc/101" "/proc/102" "/proc/103" "/proc/104"
   "/proc/107" "/proc/1074" "/proc/1086" "/proc/11")
  2> (flow (glob "/proc/*") (take 10) (map (op trim-left "/proc/")))
  ("1" "10" "101" "102" "103" "104" "107" "1074" "1086" "11")
  3> (flow (glob "/proc/*") (take 10) (map (op trim-left "/proc/")) (sort @1 : toint))
  ("1" "10" "11" "101" "102" "103" "104" "107" "1074" "1086")
  4> (flow (glob "/proc/*") (take 10) (map (op trim-left "/proc/")) (sort @1 greater toint))
  ("1086" "1074" "107" "104" "103" "102" "101" "11" "10" "1")

How about we parse digit sequences out of the path names and combine them with the originals, then sort on those digit sequences turned into integers:

  5> (defun get-nums (str)
       (flow str (tok #/\d+/) (map toint)))
  get-nums
  6> (flow (glob "/proc/*") (take 20) (csort @1 greater get-nums))
  ("/proc/11170" "/proc/11169" "/proc/1222" "/proc/1199" "/proc/1198"
   "/proc/1194" "/proc/1133" "/proc/1104" "/proc/1086" "/proc/1074"
   "/proc/117" "/proc/107" "/proc/104" "/proc/103" "/proc/102" "/proc/101"
   "/proc/12" "/proc/11" "/proc/10" "/proc/1")

csort is caching sort: it caches the result of the key function through which the elements are projected to form the argument of the comparison. In other words, so we don't wastefully call get-nums multiple times for the same path.

So basically using the shell of something like Xerox's PARC Interlisp-D and threading macros.

I would be something like this,

    (->
       ls
       myfunc
       sort -r -n
       (> list))

Followed by "Select => Execute" with the mouse, or "Execute last form" with keyboard shorcut.

Lacking it seeming helpful, it shows a largely invoked valley even I notice some beings can locate.

I guess we should distinguish "the shell environment" (which might stay Bash) with "the shell script language" (which could be any one of a number of options, just use the right hashbang header)

One that strove to do both WOULD have to do all the things you suggested as well (job management, stdout/stderr redirection etc.) So options like oil shell, nushell, fish etc.

I was just talking about "a commandline tooling language" in general, as far as use-cases. And for that, Lua (and Moonscript, which compiles to Lua but is way nicer and has been featured on HN a few times over the years) seems to fit the bill (1-based indexing aside, ugh)

I love that in Guix, Guile is used as the configuration language all the way "from the metal" (the bootloader) to the declarative environment config in userspace.

I'm one of those folks that still can't do LISPs though. If there was a way to replace some or all of the parens with significant indentation instead (which I believe Racket makes possible... but that only works in Racket), I might reconsider it.

You probably heard this one before. After a while the parens do disappear. When conventionally indented, lisp users see only the indentation and not the parens.

With structural editing, no one ever have to match parens or count parens at the end...

YMMV ofcourse.

The most interesting ones I’ve come across are:

- Elvish https://elv.sh

- Murex https://murex.rocks

- Oil https://oils.pub

Same. Do you use any of them or have you stuck with any of them? Opinions?

I’m the author of Murex, so I’ve definitely stuck with that. It’s been my primary shell for 6 or 7 years now. The only time I touch Bash is the odd occasion I SSH into a box.

However the other shells are also highly polished. I’d definitely recommend each and all of them.

Hah, that’s REALLY cool of you that you recommended other shells along with your own!!

I write a lot of JS/TS for my day job, so zx (https://github.com/google/zx) has been a nice tool for bash scripts that start getting a little too complex.

I think it kinda was one of Perl’s goals at the time. But the syntax is rather unusual.

I keep wanting to get into Raku. It has such a rich pedigree of everything learned from decades of Perl, in a clean slate. It seems to suffer from lack of adoption but it seems better than *sh in every way except for ubiquity.

It is much more complicated than Perl. Every feature you could ever want, in multiple ways, it seems like. Perl isn’t too complicated or large of a language.

Indeed Raku is a delight to use for 'shell scripting'.

Perl was originally written as an amalgamation of grep,sed,tr, awk and I am sure a few more unix utilities with their own mini languages. The idea was to use one language instead of tying together half a dozen mini-languages in a shell script. And it worked really well. Perl being a demon with text munging didn't hurt.

Raku keeps this heritage but adds so much more (for better or worse :-) ). It inherits ideas from Lisp and functional programming languages. The thing that impressed me was, how easy it was to use concurrency.

I try Perl 6/Raku every few years, so it seems like I've tried it half a dozen times by now since it started. I like some things about it, but in a way it seems like an academic project, more suited for experimentation than doing work. I always come back to Perl, which has been my favorite hammer for 30 years.

As someone old enough to have been a Perl user during its glory days, I wonder what is unusual about it, it feels right at home in UNIX.

that’s fair; maybe I’ll look at it again, it certainly has a rich pedigree at this point, even if I tend to lean towards functional languages and styles

Guess what was one of the first languages making that style popular in mainstream.

https://hop.perl.plover.com/

Have you tried NGS (https://ngs-lang.org)? I found its approach of giving you that power while remaining shell-focused to be very refreshing.

Somebody ought to package it for Linux OSs. I filed a ticket.

It has similar aims to https://www.nushell.sh/

interesting.

I've actually already written a lot of the functionality I see here as bash (and other) functions, lol (things like log, filter, debug, retry, repeat, map, fetch (a URL, with retry)... etc.). (things like: enumeration functions, table pretty-printing, etc.)

Hi. Author here.

> I've actually already written a lot of the functionality I see here as bash (and other) functions

That's exactly the reason these are part of stdlib. No reason for everybody to reimplemnt these with slight variations and copy it around.

Agreed, and good job with the tooling then!

Thanks for the kind words. I phrase it as "fully fledged programming language with domain specific facilities". Hope it makes sense.

I found https://github.com/cosmos72/schemesh lately and - once you accept LISPism - is pretty nice to use.

> Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.

If you're familiar with Python, give xonsh (https://xon.sh/) a go. It's a Bash-like shell but the syntax is Python. It has the same ease of writing shell scripts as Bash does, sans the insane language.

Been using it since 2018.

The nicest thing, I think, is that you can make a new shell command simply by writing a python function.

How does the startup time and execution speed of Python compare to other options?

(I'm not particularly inclined to Python, unfortunately, due to the dependency issues and some personally negative opinions about its design... but I like the idea of Xonsh if you're a Python person! I wish I had something like that for Elixir!)

Obviously slower, but since when is the speed of a Bash script a critical factor?

Lua is my favourite, too, but i found writing in awk to be more pragmatic.

So Awk for everything (or in my case gawk or frawk)?

Or just text-processing things, where it's most suitable?

(I think Awk is underrated, for sure!)

We've transitioned all of our Bash scripts over to Ruby and are pretty happy with it. We almost exclusively use only Ruby's standard library. 'fileutils' and 'rake' have a bunch of nice functions to replace Bash scripts.

See, FileUtils for things like rm_rf, rake provides sh which can execute commands, then there's FileList and Pathname. It even comes with its own template engine (erb) if you need that. Regexes as first class citizens are awesome for shell scripting.

I can also recommend using optprase + ostruct for commandline argument parsing.

I usually go with python when my .sh/.bat script files gets too complex. So I guess we wento down the same road.

Like ruby (I guess), python is very pleasant to work with for small scripts and it's portable between linux and windows, which is really cool (at least in my use cases)

In python, glob, Path and subprocess are mostly all I need...

I moved to using perl first, but these days I'm mostly using python except where regular expressions are needed in which case I'm still using perl

I find python is not a natural fit for shell script replacement (may be I am doing it wrong).

Executing a process and bringing back status, stdout and stderr seem tedious. While in Perl, it is as natural as it is in a bash script (well almost).

Raku on the other hand is _Fun_.

coming back to raku after basically any other language is a breath of fresh air. i haven't even been worrying much about when the ast stuff lands cause i can't think of anything i would even want to mess with that much, even though i seem fundamentally incapable of engaging with lisp/prolog/forth without giving in to the siren song of metaprogramming. to me that's a very strong sign of how much it gets right as a base language.

i do all my automation with one tiny 20 line raku script that manages a thousand things. hell, the entire monitoring site for our frog is basically just a pile of mixins with a neat little pipeline interface unifying everything from request profiling and html templating to reading from and filling that cache in parallel whenever it remotely pulls the sensor data off various things and gets the current webcam frame.

Same but I found it a really bad choice for a Bash replacement

You have to reinvent the wheel every other day since the standard library doesn't come with much included

I see Penlight (which is Python-flavored, but also has some functional programming constructs) https://github.com/lunarmodules/Penlight , Moses if you like functional programming https://yonaba.github.io/Moses/, likely others, but you're right in that there's no standard, "opinionated" stdlib. No bigint support on LuaJIT etc.

I'm an incredibly happy user of nushell, which brings all the best features of other shells terse pipelining syntax and all the best features of more well designed scripting languages (functions, modules, lexical scope, data structures, completely optional types) in one awesome package that also comes with editor (LSP) support and excellent documentation

https://www.nushell.sh/

(The intro page may be a bit misleading. You can freely mix-and-match existing, unstructured as well as nushell-built-in structured commands in the pipeline, as long as you convert to/from string streams - its not mandatory to use the structured built-ins. For example if an existing cli tool has json output, you can use `tool | from json` to turn it into structured data. There are also commands like `detect columns` that parses classic column output, and so on - the tools to mix-and-match structured and unstructured data are convenient and expressive)

Some highlights:

- automatic command line arguments and help by defining a main function and adding comments to each argument - e.g. https://github.com/nushell/nushell/discussions/11969

- run commands with controlled parallelism: https://www.nushell.sh/commands/docs/par-each.html

- easy parsing of raw input https://www.nushell.sh/commands/docs/parse.html

- support for a wide variety of data formats https://www.nushell.sh/commands/categories/formats.html

- built-in support for talking to SQLite databases: https://www.nushell.sh/book/loading_data.html#sqlite

edit: it looks like Mitchell Hashimoto was recently impressed too https://x.com/mitchellh/status/1907849319052386577 - rich functional programming library that blends with pipeline syntax https://www.nushell.sh/book/nushell_map_functional.html

Addendum: Its not my login shell. I run it ad-hoc as soon as the command pipeline i'm writing starts getting too complicated, or to write scripts (which of course can be run from outside nushell too, so long as they have the correct shebang)

NetBSD's base system includes both the best scripting shell, ash, and lua. Thankfully, no python, no startup cost.

if bash/sh scripts get too long (or long-lived), I switch to python when possible.

As a minor example, I hate having to remember (or explain) ${var##pattern}

It is useful for manipulating paths, but it is cryptic.

in comparison, something like:

  newfile = os.path.join(newpath, os.path.basename(oldpath))

IMHO is more self-explanatory and robust.

Also, argparse is the best.

That's because you don't use the native lib pathlib from python ! It was a delivery when I knew about it. For example, to join paths, you can do so much easily, see the section "operators" of this link : https://docs.python.org/3/library/pathlib.html

Reply on news.ycombinator.com