I've been sensing a stigma against shell scripting increasingly over the past decade. It's nice to see bash being used in creative and elaborate ways like this.
Bash should be considered an esoteric programming language. Shell Programming Considered Harmful.
It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort.
(that said, I understand the appeal of doing weird things with it, like the Minecraft server, for the same reason other esoteric programming languages have appeal)
Ideally we should all try to more programming languages that are harder for us to shoot ourselves in the foot with, rather than easier ;-)
"I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular."
Consider this:
for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done
In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.
for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done
For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo.
https://www.gnu.org/software/bash/manual/html_node/Shell-Par...
lee@sp3tmp1:ls *.png
'a b.png' c.png
lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done
convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924.
convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229.I think you could be making a more complicated point.
You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.
Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0.
I have covered some of this here:
https://www.linuxjournal.com/content/parallel-shells-xargs-u...
(Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)
I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory.
The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.
The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced.
This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline?
Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.
All else should be quoted, and newlines can happen, primarily from abuse. Do not trust user input!
Calling fopen() on a file with every ASCII character will work on POSIX, as long as it does not contain a null or forward slash (/). Coding defensively means that this does not crash your processing.
I was objecting to the knee jerk inclusion of warnings against file names including newlines. Has anyone encountered one in the wild that was not itself an error?
>> Consider this:
> for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done
This is buggy, and it actually represents exactly the problem with Bash scripting.
The problem usually come when you attempt to use it for nontrivial programming. But that just not what it's for, any more than the difficulty in building an OS kernel is a legitimate complaint against PHP.
This is important to know in the context of what you are saying.
The Algol structures in the Borne shell are the most incongruous parts of UNIX.
quick: without looking anything up, write something in bash that takes a string of comma-separated values and turns it into an array of those values.
then check to see if your answer is on this very long list of wrong implementations:
I would just use Python, where strings are sane and lists are sane: my_string.split(", "), boom, done, no jumpscares.
I disagree. Bash is fine. Nothing esoteric about it.
> It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort.
It's easy to program, but it has a learning curve. Use shellcheck. It's OK to go first to bash if you need a shell script. Other languages make pretty crappy shell scripts.
Arrays:
- `${#myvar[@]}`: unreadable
- `mapfile myvar < <(grep pattern file)`: "mapfile" is not exactly a clear term (there's the synonym
"readarray", but "mapfile" is the reference; worse, you may find both used inconsistently); the
command as I wrote it also has two problems
- `${myvar##.*}`: unreadable
- `echo $(IFS=,; echo "${myvar[*]}")`: unreadable; most also don't know the difference between `[@]`
and `[*]`; this doesn't also work for two-char join operations
- `[[ $myvar =~ $(echo '\bpattern\b') ]]`: unreadable, and most don't know why command substitution
is needed
- `3>&2 2>&1 1>&3`: most don't know what this meansThe main issue is that many people try (by necessity or not) to cobble scripts together without really learning the language and its idioms. It's not that hard to really learn it though, it's basically one man page then the mandatory [wooledge wiki][1].
It may be much easier (and most of the time I would generally prefer) to use Python for scripting purposes, but Bash is a) almost always available and b) widely portable.
If you manage any servers where you never know if Python is available or if it’s version 2 or 3 (and you have no power to change this for one reason or other), you know the value of this. No, it’s not ideal, but we live in a non-ideal world. Long live Bash.
And when you consider there really isn't a standard library, mostly gee-I-hope-this-UNIX-util-is-installed...
For example, I'm working on a Hashicorp Vault server install and due to it's sensitive nature, I'm strongly considering using bash to set it up over something like python. I love the ability to test in a robust language, but the extra dependencies are hard to justify, at least on a security sensitive application.
Well, we "write" one-liner programs with it all day every day - since it's the shell.
I'm not saying bash doesn't have a bunch of warts, but - what other shell language would you suggest as an alternative to it?
Don't get me wrong, I have nothing against fun and creative projects. If someone wants to script something complicated in bash for the lulz, the challenge, to learn something, or just to show off bash-skills, I'm all for it.
What I am against, is when I get asked to debug a 2000 LOC system used in production that was thrown together in bash, waste 2 days on it, only to end up doing what should have been done in the first place, and rewrite the damn thing in 200 lines of Python or 400 lines of Go.
It can, and should, be used to automate some things, string some commands together, make certain tasks easier. I use and write bash scripts in my daily work, and make my life easier with it.
But there comes a point, and that is sooner rather than later, when something should no longer be written in bash, but in a language that wasn't designed primarily as a command line interpreter. Can I make large complicated systems in bash? Sure. Can I carve the handle of a hammer so that it can be used as a screwdriver? I'm sure someone more skilled in woodworking than me could do it. But sometimes the question isn't if it can be done, but if it should.
Shell scripts to automate processes with other tools is exactly what they are designed for, and adding in other logic and loops is fine too, but if you are setting out to create what would be considered a full program rather than an automated task you should definitely consider alternatives (of which there are several suitable choices) that usually end up being easier to maintain.
It used to be that the main alternative was perl, but frankly that was/is no better (IMO much worse) than bash for encouraging the generation write-only (difficult to maintain) code. Python is often a good option, and I've seen node used, though these don't have the ubiquity that sh/bash does (or perl used to have) which is one advantage of bash: it is almost everywhere, even Windows these days (or stick with core posix sh and you'll find support a little wider again), though maybe without some of the supporting externals which is when tricks like these come in.
If you are writing some simple automation, or maybe slightly less simple, then bash is fine as a go-to, but if you are setting out to write something complex bash is a bad choice unless compatibility with many environments you can't dictate have another interpreter present is a non-negotiable requirement.
There is the grey area when a script, or collection of scripts, becomes a larger program/environment over time. Sometimes the effort to rewrite (and fully test) in another environment is not worth the small benefit of doing so, and is therefore a bad use of developer/admin time that could be used elsewhere, particularly when high performance is not an important feature. Though if you find yourself with a lot of bash script, do yourself (and your later replacement when you've moved on to newer projects) a massive favour and make sure it is well documented even if that is only in in-script comments.