Shell script mistakes
pixelbeat.org
pixelbeat.org
That bit, writing a script that generates a script, happens surprisingly often in bash. It's cheaper to pipe a stream to sed that converts it into a shell command than it is to iterate over all the lines, and individually pluck out the arguments for the commands you want to execute. Leaving the script as something that outputs shell commands also lets you inspect what it does before committing to it (by piping it to bash).
for fic in **/*(.); do
doSomethingWith $fic
done
for fic (**/*(.)) doSomethingWith $fic
Shell script syntax has always been hard to remember to me (but I feel it to be less than intuitive compared to lisp, perl, js, c, ruby, python...) , subject to different behaviors between shells (csh vs ksh IIRC), string manipulation is close to non-existing, types are more difficult to use than in PHP,...
Okay, it works, we can do stuff with it, we can hack a quick thingie there and there,... I know the main appeal is, "it just runs everywhere, no install required" (minus the csh/ksh differences), but is it really the tool we need for all we use it for?
I write for and "while read" loops on the command line a dozen times a day, () subshells, process substitution, sed, awk etc. The more proficient I am on the command line, the more I am with shell scripts, and vice versa.
Another advantage; the shell is a repl in which you can prototype your script. Perform the operations manually, pull the commands out of your history, and adjust as necessary.
You earned my respect.
I like REPLs for small operations. When talking about a "full" script, I'm more at ease in a text editor with a script I'm gonna adjust as I run it. Best example is a syntax error: the script just won't run. Meanwhile, in a REPL, I may get myself stuck without knowing it.
If I actually need to do something with an input or output before sending it to the next command, I'll switch to Python.
For CLI tasks in Python, http://docopt.org/ and http://shell-command.readthedocs.org/en/latest/ are invaluable tools. Yes, I know about Envoy, but shell_command is much nicer to use in my opinion. Envoy doesn't automatically escape arguments, which is annoying for all the reasons that SQL injections are annoying.
If it's not something that can be done trivially with sed or awk, I'll consider whether it's something that can be implemented in a generic way to be used elsewhere, in which case I'll write that specific utility, and insert it into the pipeline (with appropriate arguments). Only if it's fairly specific, or I need to package it for distribution, will I consider trying to rewrite everything in a single language script.
pip install python-crontabFor me, the point at which Perl makes more sense is if your script requires more internal logic than it does depends on spawning other programs.
For example, if I'm writing a routine to auto snapshot ZFS / Btrfs volumes and delete any over a certain age, the script would be dependant on your file system CLI tools. So it makes more sense to have an 80+ line shell script than it does to write that in Perl / Python.
However if I was writing a routine which requires users inputting details, where those details need to be sanity checked and then stored some where (such as a database), then the core logic of that program resides within your script (where you'd have to read inputs, do your sanity checks and then write to the database). So a Perl or Python script makes more sense.
Obviously you can do either of those examples in each of those languages (crudely speaking as I know shell scripts aren't technically a programming language); but that's just a basic example of where I personally draw the line.
I also think this is one of those occasions where it doesn't massively matter which approach you take just so long as the code works and is maintainable (though I draw the line at one maintenance script I saw last year. It was a Python script where every other line was os.system. It just struck me as rather pointless starting a Python interpreter if you're just going to use it like a shell script - you might as well do the whole lot in the shell to begin with).
*(though I draw the line at one maintenance script I saw last year. It was
a Python script where every other line was os.system. It just struck
me as rather pointless starting a Python interpreter if you're just
going to use it like a shell script - you might as well do the whole lot in the shell to begin with).*
Of course they are.
To the article, I've never used a shell that doesn't understand [[ (guaranteed to be builtin, not sensitive to the hyphen issue) and {,} expansion.
For the "for file in *; ... " that is also susceptible to the shell arg limits and is fairly hairy.
Lastly if you're concerned about performance -- well, it's definitely time to leave shell for something better.
That may be, but that doesn't mean they aren't pretty widespread. [[ and {,} are bashisms. However, they still work when bash is executed as /bin/sh, as is the case on RedHat and some other Linux distributions.
However, other distributions, including Debian, Ubuntu, and any embedded system using Busybox, point the /bin/sh symlink to a version of ash, the Almquist shell. It is used for scripts instead of bash because it is much smaller and faster (reportedly, Debian/Ubuntu boot speed improved significantly -- leaving the shell isn't a very good option for sysvinit).
(Funny; I could've sworn I used {} in ksh under Solaris but apparently not!)
Your point stands though, they are bashisms for all intents and purposes on Linux.
As to speed -- well, again, I think if you want speed don't use shell. For startup you're better off redesigning the whole damn thing (cue systemd). Changing shell is like overclocking your cpu when you should be choosing a better algorithm.
for $file in *;do wc -l $file;done
for $file in *; { wc -c $file ;}
Is the for loop even necessary?
echo wc -l * |sh