Hacker News Reader

Comment by teddyh | Hacker News Reader

It’s amazing how much of a superpower merely reading the manual is nowadays.

<https://www.gnu.org/software/bash/manual/bash.html#Arrays>

While I agree with the larger sentiment I think you’re making (I also make it a habit to peruse the manual just to see what is available), how often do you reread the manual, especially for your shell? I knew about associative arrays in bash, but not by reading the manual as those were introduced in v4 and I’ve been using bash for longer than that.

You should probably read the release notes for new major versions. Also, anytime you think “I wish there was <feature> available in this program”, you should probably double-check if such a feature has appeared since you last learned the program.

That's how I discovered the '&' regex feature in less some years back.

Do you mean PCRE’s “(?&name)” feature? That is not portable; for instance, Python calls it “(?P=name)”. Or do you mean the common feature of “&” in the replacement string meaning “the whole matched string”? While this feature is common in languages which use regexps (including ed, sed, awk, etc.), that’s not actually a regexp feature, since the & character has that effect in the replacement string, not in the regular expression. It is also not always portable; for instance, Emacs requires escaping as \& for the character to have the special feature.

From man(1) less:

  &pattern
       Display  only  lines which match the pattern; lines which do not
       match the pattern are not displayed.  If pattern  is  empty  (if
       you  type  &  immediately  followed  by ENTER), any filtering is
       turned off, and all lines are displayed.  While filtering is  in
       effect,  an  ampersand  is  displayed  at  the  beginning of the
       prompt, as a reminder that some lines in the file may be hidden.

  $ less --version
  less 487 (GNU regular expressions)
  Copyright (C) 1984-2016  Mark Nudelman

Oh, you meant the program “less”! Right, I understand what you meant now.

:-)

Yeah.

I was viewing some files and thinking "it'd be really cool if less(1) had some way of filtering to just lines matching a specific pattern".

RTFM'd and TIL'd.

Mind that at that point I'd used less for ... oh, at least 30 years.

Agreed. This is also how I found out that features in Python I took for granted, like breakpoint(), were much more recent than I thought (3.7). Nothing like having to use ancient versions of a language to discover limitations.

I do a fair bit of bash scripting and there is no way I can remember all the features and syntax I use, so of course I read parts of the manual regularly and often learn new things. You don't need to ever read the whole thing.

I once worked with a dev (frontend, of all things) who reread the bash manual annually.

I work in ops, and I don’t even have that dedication. My hat is off to them.

TBF, the Bash manpage now runs > 80 pages:

  man -Tps bash | ps2pdf - bash.pdf

That can be daunting. Though illuminating.

Bash is low on the list of things to learn, especially as many greybeards suggest keeping on POSIX compatibility and/or using a "proper" language (Python, older: Perl) for anything longer than a few lines.

They suggest that based on their experience. Error handling in bash is awful so I advice against using it for anything complex.

Anything complex should be written in a competent language like Java. Script languages (like Bash and Python) are for short (a few lines long) scripts. Using the tool outside the scope of what it was designed for is not a good idea.

Tell me you have never seriously used Python without telling me you have never seriously used Python.

I mean, viewing Python strictly as a scripting language? I am honestly lost for words. There are many huge and major applications and web sites written in Python, without people regretting it after the fact. And yet here you are dismissing it out of hand without a single argument.

Meanwhile most of the time topics like this come up and people hate on shell scripts, those of us that like them see those criticisms the same way you're looking at this comment about python: So far out there it's almost not worth responding. I think that's why GGP and GGGP think "greybeards" don't think it's worthwhile based on experience - it's actually not worth arguing against misinformed comments so newer people don't realize it's still heavily used, just quietly in the background for things it's actually good at.

Further down is a comment about that: https://news.ycombinator.com/item?id=42664939

> …not worth arguing against misinformed comments …

Yeah, I have these same response patterns. Shell works really well for some use cases. I generally don’t respond to the comments that list the various “footguns” of shell, or that complain about security holes, etc. My use cases are not sensitive to these concerns, and even besides this, I find the concerns overstated.

Don't be too rude, this is a common view among people who are technically adjacent but not engineers, like IT people. It's an incorrect superstition, of course, but in tech almost everybody has their superstitions. There's no reason to be rude -- ignorance is not a crime.

I see that kind of thing all the time. Usually it is about static types. People think that dynamic languages aren't "serious", or something. It is laughable that these people still make up a significant amount of comments, here in 2024.

Why shouldn't you use Python for larger projects, and why do so many startups succeed with large Python repos?

When my scripts outgrow bash, they almost always wind up in Python.

That said, Sonnet 3.5 had gotten me much further in bash than was possible before - and it's all really maintainable too. I highly suggest consulting with Sonnet on your longer scripts, even just asking it "what would you suggest to improve".

Using a tool beyond its design can be problematic.

But Python is not designed to only be a scripting language:

> What is Python?

> Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. It supports multiple programming paradigms beyond object-oriented programming, such as procedural and functional programming. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++. It is also usable as an extension language for applications that need a programmable interface. Finally, Python is portable: it runs on many Unix variants including Linux and macOS, and on Windows.

You can go a really really long way, with a script that will work everywhere, with just set -e and aborting at the first error.

Better yet, 'bash unofficial strict mode':

    set -euo pipefail
    IFS=$'\n\t'

That has lots of issues. See https://mywiki.wooledge.org/BashPitfalls#set_-euo_pipefail for some.

It does, yes, but IME if you can get people who aren’t familiar with shell to use it, it prevents more problems than it solves on the whole.

Then again, so does enforcing passing shellcheck in CI.

One can made arguments both for and against `set -e`.

Over may carrier I've seen many shell scripts:

The most common case is no `-e` and no explicit error/status checks for most commands. It's unreliable and such scripts is one of reasons why shell got it's bad reputation (and criticized by coders using languages where an unhanded exception would terminate a program).

In my own scripts I often use `-e` (but not always as it is a tradeoff). If you unfamiliar with shell `-e` also could cause problems. But `-e` is not magic which makes a script better, one have to know how it works.

The option author of this article advocates (no `-e` but explicitly check every command which could fail) is the least common IMHO. Unless the scripts calls just 1-2 commands it's tedious and the resulting script is harder to read. May be you can find such style in some opensource projects but I've never seen such scripts at work.

Give bash some credit - it's actually amazing for even very large, complex systems (of which there are many -- quietly doing their jobs for decades.)

With xargs or GNU parallel, you can have multi-processing, too. Combining with curl or ffmpeg, you can literally build a production-grade web scraper or video transcoding pipeline in a couple of minutes and a few dozen lines of code.

I write a lot of shell and my advice is don't use plain POSIX shell. Write bash.

It is 2025. bash is present almost everywhere. If, by chance, your script is run somewhere that doesn't have bash then guess what?

Your POSIX script probably won't work anyway. It will be a different system, with different utilities and conventions.

Line count is not a good reason to choose or avoid bash. Bash is quite reliably the lowest common denominator.

I dare say it is even the best choice when your goal is accomplished by just running a bunch of commands.

All that said, bash is gross. I wish we had something as pervasive that wasn't so yucky.

My one-shot memory worsened with years but grasping concepts from examples got much better due to experience. So I find walls of text much less useful than a continuous “synopsis” of snippets and best practices. One usage example is worth a hundred of words and few pages of it may be worth a whole manual, especially when “at hand” through editor means. For bash idk if that exists on the internet, so I maintain my own in my one big text file. I wish every manual had the synopsis-only section (maybe I should just LLM these).

> One usage example is worth a hundred words ... For bash idk if that exists on the internet, ...

Totally agree with that - I maintain a big txt file too.

Maybe this bash compendium is a bit similar:

https://www.commandlinefu.com/commands/browse