Bash $* and $@ (2017)
eklitzke.org
eklitzke.org
Even then, my threshold for “this should be Python” has shrunk over the years: I used to say “greater than one screen of code” but now it’s more like “>1 branch point or any non-trivial scalar variable”.
I have done something similar too once in a shell-like Python CLI I worked on. Can also use __ror__ to be able to pass primitives into your commands like `[1, 2, 3] | SumCmd() | PrintCmd()`
I do agree that typing something like that, especially when working in a shell, is much nicer than having to go back and forth all the time to type `print(sum([1,2,3]))`
Here it is mentioned in a talk about Python Aesthetics by Brandon Rhodes: https://www.youtube.com/watch?v=x-kB2o8sd5c&t=8m24s
I'll stick to Python, but Ruby is worth a look.
It just challenged my colleague to make the code denser, to stay within the given limit. Sigh.
Although I think all of these things are just complicated ways of saying "please seriously reconsider writing it in bash."
But I write bash scripts all the time, I just try to keep them as short and simple as possible.
It's still better to script in Python or Ruby than Bash. Nobody understands Bash. It's even more mysterious than Perl.
1. more emphasis traditional programming paradigms (be it JS, LISP, Python, whatever) which leaves a platform that is arguably a better designed language but however is a poorer REPL environment for it. Bash works because it's terse and terseness is actually preferable for "write many, read once" style environments like an interactive command prompt.
2. or they spend so much effort supporting POSIX/Bash -- including their warts -- that they end up poorer scripting languages.
I think what we really need isn't to rewrite all our shell scripts in Python but rather better shells. Ones which work with existing muscle memory but isn't afraid to break compatibility for the sake of eliminating a few footguns. Shells that can straddle both the aforementioned objectives without sacrificing the other. But there doesn't seem to be many people trying this (I can only think of a couple off hand).
Not that I'm taking anything away from zsh. It is a nice shell. But I think we can do even better considering how dependant we still are on shells for day to day stuff.
I agree with the idea of breaking backwards compatibility, but Powershell honestly has enough core design issues that it itself is starting to feel like it needs a major backwards-incompatible update.
Powershell 7 has a full Win32.
I mean, I would expect the syntax to be something like:
make >/dev/null 2| more
Note that the order of redirections
is significant. For example, the
command
ls > dirlist 2>&1
directs both standard output and
standard error to the file dirlist,
while the command
ls 2>&1 > dirlist
directs only the standard output to
file dirlist, because the standard
error was duplicated from the
standard output before the standard
output was redirected to dirlist.
Can it be memorized? Yes, because it is just a single 'incantation': "2>&1". Just put that redirection operator before the redirection of the standard output to the file, and the result is stdout goes to the file, stderr goes to the pipe.
"Redirecting streams" winds up being confusing - it's all just `dup2`.
Note that the order of redirections is significant. For example, the command
ls > dirlist 2>&1
directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command
ls 2>&1 > dirlist
directs only the standard output to file dirlist, because the standard error was made a copy of the standard output before the standard output was redirected to dirlist.
Yes, I know that underneath it's all calls to `pipe()` and `dup2()`, which I can do (and have done) in a language other than shell. It's the shell redirection syntax (for anything more complex than simple redirection or a pipe) that just doesn't make sense to me.
command 2>&1 > >(cat - | cat - > fd1) | cat - | cat - > fd2
How to read this:
When Bash sets up the file descriptors for 'command' it initially makes descriptor 1 refer to the pipe, and descriptor 2 refer to the terminal.
So, the dup operator (2>&1) copies the fd in descriptor 1 (which is the pipe) into descriptor 2 (so after the dup operator is processed both stderr and stdout for command reference the pipe).
Next descriptor 1 is replaced (the > operator) by a reference to a fifo created by the process substitution operator (the >(...) operator).
So now command's descriptor 1 refers to the fifo created by >() (which itself contains a "string of commands").
Then, because stderr for command was made to be a copy of what was previously stdout (before modifying stdout) it continues to refer to the pipe Bash setup (the | operator), and stderr now flows out over the pipes to another "string of commands".
( echo "stdout"; echo "stderr" 1>&2 ) 2>&1 1>/dev/nullBest of all though, it's absolutely compatible wherever you need to run it.
"If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date."
Cuz I've never heard it until this thread, where it pops up multiple times. Great phrase but went 0-60 in usage.
This is true, especially of Python scripts.
Edit: Getting some downvotes, so to clarify, I am indeed saying that concerting something from Python to Perl has a high likelihood of making it for less maintainable. I get that people can write good perl. As someone who has had to maintain perl in the past; the fact is that it's far more common for the end result to be horrible perl. I have some issues with Python, but it is FAR more maintainable than perl.
Every single person who doesn't use Perl every day, professionally knows a different subset of Perl. This is one of the few times where being more knowledgeable than the interviewer is also a problem. You will write something, and the interviewer will question you on it because he has never seen it.
I finally solved this problem by bringing one of my personal programs written in Perl and also rewritten in Python so we could talk about it. Now, the interviewer is in MY subset of Perl AND can't argue because I have working Perl code in front of him. This was back in the 1990's when everybody in VLSI design expected you to know Perl.
Because of this "different subset" issue, if you want to maintain a Perl script, you have to basically know the entire language. This is what makes maintaining Perl scripts so difficult.
This "different subset" problem is the whole reason I left the Perl ecosystem back in 1996(!) at the height of Perl's popularity and never looked back.
Perl5 still works fine, and is maintained.
> Is it a worthwhile replacement or too little too late?
False dichotomy - see above.
I have actually found that is pretty much always a problem. If you pull out something the interviewer is unfamiliar with they will often assume you are full of it. I have had such people refuse perfectly good explanations for things they haven't heard of because they assume incompetence before that possibility.
I'd take those experiences as a blessing rather than a curse.
Last I knew, that position was still open almost a year after the fact.
Has this changed at all?
While I still regard myself as a vastly better VLSI designer than programmer, my ability to wrangle software the whole way from assembly language on a chip to just shy of the top of a full web stack pays far better than my ability to wrangle transistors. And, in my opinion, attacks far more interesting problems.
At work lately, there's been a spate of contorted "just why?" Python scripts that could've been accomplished elegantly in a handful of lines of shell. While no one would select shell languages as the ideal for a lot of complex logic, data parsing, etc., there's no competition for a shell when you need to do what shells are meant to do: chaining invocations, gluing and piping output, and so on.
I'm not sure about that. Google certainly has some very talented people among their developers, but most of them are so bad at programming that they had to invent their own heavily watered down programming language.
That's where we came in with the need to use "$@" vs the unsafe-by-default bare $@.
There is no reason this shouldn't be easy to do in Python too, if it is then it is something a library should fix.
shellcheck /scriptsdir/script
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang.
shellcheck -shell=bash script
Unknown shell: hell=bash
Although there may be some irony in the fact it cannot sort out it own argument parsing.
1. I do not use other scripting languages such as Python, Perl, Ruby, etc. That means, e.g., for quick and dirty one-offs and prototyping, I can omit the shebang. Debian's "dash" scripting shell is derived from NetBSD's ash, the one I choose for interactive use.
SC2148: Tips depend on target shell and yours is unknown. Add a shebang.
I always have the top of my shell scripts with a shebang, even if the script isn't intended to be directly executed.
Pick the user's bash from PATH environment:
#!/usr/bin/env bash
#!/bin/bash
#!/bin/sh
#!/usr/bin/env python3
#!/usr/bin/env vi
My point here is it is less to due with the language and more to due with the mindset when solving a problem.
The main issue I see with more inexperienced devs with bash is that they tend to think it’s okay to be lazy with the code because it’s just “bash”. If you would write safety checks and comments in your python you should be doing the same in bash really.
This, I say as someone who loves Python and I use it as my primary language both privately and at work. But I have to admit, Python scripts do not really age well.
Before that, well it's been a while. But with and yield were added as a keywords (breaking any code using them as a variables). The xreadlines module disappeared at some point (I'm sure a few other modules have died along the way). At some point you couldn't raise anything you wanted as exceptions any more. Oh, and the source encoding declaration. That's what I remember off the top of my head.
Minor things that are handled in things that are maintained, sure. But many things are also built once and then expected to run for a long time - especially in the role shell scripts usually fill.
Note, that I don't argue that any of these changes were bad. They were all good changes that made the language better. But, they resulted in old code breaking. One extra headache to deal with for the poor schmuck that was responsible for upgrading this server or that to the latest OS release.
I'll provide the counterexample.
Any scripts I wrote more than 10 years ago for actual "Bourne shell" /bin/sh often fail in mysterious (and sometimes silent) ways when run under bash.
Bash is just as guilty of backward incompatibility as Python is. Python at least generally has the decency to squawk about it.
Over the years I've spent way more time going into inconsistencies about the various platform tools (e.g. GNU vs. BSD implementations of sed, grep, find, etc. even before you ge to the space aliens-with-broken-translators realm of AIX) — which using Python avoided needing to care about — or library / file naming changes. Similarly, a few years back there was way more time used when various HTTPS improvements flushed out code using old versions of OpenSSL or, worse, gnutls.
https://www.python.org/dev/peps/pep-0466/
(Can't remember the details, but it may have been something about some extra hoop you'd have to jump through to trust self signed certs that broke our testing infrastructure)
I know everybody likes to think devops and cattle/pets and "you should never ssh into machines" are how things should be and there thats how they are, but in the real, non-sv software startup world sysadmins around the world who get that 3am call are fixing some devs shit with bash and sysv/systemd scripts.
I feel at this point it's just a bandwagon people jump onto because they want to feel superior. Just mention bash on HN and expect any number of "... don't use bash" comments.
Bash best practice is always double quote variables! Do that and the post becomes rambling about what happens when you dont follow standard bash practices.
[ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “click”https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
Bash also incorporates the POSIX shell, but has extensions. This is fine if you are using it, but if you are writing a script which may need to run on another system, its better to keep it to POSIX.
(edit: URL - thanks userbinator)
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
[0] https://github.com/anordal/shellharden/blob/master/how_to_do...
- You can write @ARGV instead of "$@".
- You can write @myarray instead of "${myarray[@]}"
(Related: Thirteen Incorrect Ways and Two Awkward Ways to Use Arrays https://www.oilshell.org/blog/2016/11/06.html )
Example:
oil$ var myarray = @('has spaces' foo)
oil$ var s = $'has\ttabs'
# function to print an array element on each line
oil$ lines() { for x in @ARGV; do echo $x; done }
# pass 3 args -- 2 from myarray and 1 from s
oil$ lines @myarray $s
has spaces
foo
has tabs
As I see it, the goal for a project like Oilshell (a shell with both a new syntax and support for standard bash syntax) would be to replace bash as the default shell in distros. Until then, Oil scripts lack the primary feature of bash scripts just like other alternative shells.
Right, that's the goal of Oil.
Most if not all my shell scripts are just piping executions of external commands. I find all the code needed to properly run a process and process its output is much easier with the UNIX toolbox and a couple of pipe commands, than having to handle all those input/output buffers, command execution modes, etc in any other shell script language.
OTOH Plumbum [2] has been mentioned here, and it seems fantastic for that use case. But I think the issue is obvious, in that it took a conversation in HN to raise awareness of this tool: it is not officially promoted, or recommended even, as the solution for replacing shell scripting, so it is kind of obscure (unless you are actively into the language or somehow by chance end up getting to know about it, that is)
There is also the thing about choosing Python to replace Bash scripts would force having to install Python in all of the project's Docker images, while a short POSIX script works as-is.
[1]: Saying Python because that's the most common suggestion for replacing Bash.
If you know enough bash to disagree, you know enough bash to use the third case safely :)
# ARGV=( "one two", "three four" )
# CORRECT
su root -c 'rm "$@"' -- "$@"
rm "one two" "three four"
# incorrect
su root -c "rm \"$@\""
rm one two three four # wrong arguments
# incorrect
su root -c "rm" "$@"
rm # -c doesn't use arguments
# incorrect
su root -c 'rm "$@"' "$@"
rm "three four" # loses the first argument (?!)
# incorrect:
su root "rm" "$*"
rm one two three four # wrong arguments
# incorrect:
su root "rm $*"
"rm one two three four" # command not found
# CORRECT
ssh remote -- 'rm '"$(printf '%q ' "$@")"
rm "one two" "three four"
# incorrect
ssh remote 'rm '$(printf '%q ' "$*")
rm "one two three four" # wrong arguments
# incorrect
ssh remote rm "$@"
rm one two three four # wrong arguments
# incorrect
ssh remote "rm \"$@\""
rm "one two three four" # wrong arguments
# incorrect
ssh remote 'rm "$@"' "$@"
rm one two three four # wrong arguments
# incorrect:
ssh remote "rm" "$*"
rm one two three four # wrong arguments
# incorrect:
ssh remote "rm" "$*"
rm one two three four # wrong arguments
I was wondering if you could elaborate a little bit on what you mean by:
> as you're locally specifying the args without any further IFS interference
I know that IFS = Input Field Separator. But how does it interfere when doing su root -c “command”? I might be missing something obvious here.
Also the ssh + printf is gold! But to be honest, I would personally never use anything from $@ inside an ssh remote “command”. Too risky even with proper quoting.
Thanks!
I’ve been bitten by $@ and —- related mistakes at roughly the same time within the last month or so. Luckily nothing to do with sudo.
# CORRECT
su root -c 'rm "$@"' -- "$@"
rm "one two" "three four"
You could fake this with printf %q if you tried hard enough but that’s a high-risk game to diagnose issues with, for example when you’re trying to figure out why newlines are reaching the command as the letter n.
su root -c 'rm "$@"' -- "$@"
The second instance is the argument list being given to su by the shell running the su, and it is just normal "$@" semantics there. One has to realize here that the second "$@" is being expanded by the current shell (the one running su) while the first "$@" is not expanded by the current shell, but is instead expanded by the spawned shell.
The "$@" is needed twice, because two expansions ultimately take place, the first expansion occurs in the current shell, the second one is delayed and occurs in the spawned shell.
The first expansion is in the first process, second expansion in the child. So $@ number two is the input to $@ number one.
#!/bin/sh
ssh remote "$*"
./ssh-remote.sh cat your-favorite-file.txt
I was mistaken about su / sudo.
A short list of possible problems (of course depending on the shell in question):
spaces in filenames
newlines in filenames
nonprintables in filenames
empty variables and their expansion ([ x$foo = "xsomething" ])
errors in pipes
environment madness
/bin/bash ?= /bin/sh
Arrays or the lack of it
Space separates lists as arrays
#!bash vs. #!/bin/bash vs. #!/usr/bin/env bash vs. #!/usr/sfw/bin/bash vs. ...
Unwritable and unreadable control structures (if [], case, &&,...)
Information leaks via ps
and many others...
Never use shell except to search for and invoke a sensible language. And anything is more sensible, including C, Perl, brainfuck and Basic.
There are quite a few pitfalls in shell scripting. You can considerably reduce them by limiting yourself to only being compatible with modern versions of bash and settings things like pipefail, nounset, etc etc.
I do agree that in general a good programming language will be a better option.
> anything is more sensible, including C, Perl, brainfuck and Basic
I do disagree with that however. A 5 line bash script may be 500 lines of C, will take a hundred times longer to write, and may contain memory safety issues (which the bash script at least wouldn't).
I know brainfuck is hyperbolic so I won't argue against that. Something with no filesystem or process forking abilities obviously can't be used for any real task.
I think perl and basic have just as bad syntax as bash though, if not worse. Basic's penchant for "GOTO" is awful, perl's syntax as a whole is just as peculiar as bash's in many places.
I guess my overall point is that bash is usually not a good option compared to modern languages, but it's a darn sight better than you give it credit for. I think it still has its place for 5 or 10 liners that are easy to express and read in bash and don't need any abstractions beyond what coreutils provide.
Basic does have Goto, but modern dialects do have all the usual control structures. Perl has weird syntax, but far less dangerous footguns: e.g. there are proper arrays, as opposed to many shells. One can distinguish between an empty and an undefined string. One can declare variables and there is the notion of data types. There are even things like taint mode. In shell, you can't even properly iterate over a directory without nasty surprises.
Same in C. Yes, there are memory safety problems, but those are outnumbered by far by shellscripts exploitable via some expansion or variable injection. Its just that thankfully nobody uses shellskripts as network services, so you don't see as many reports about that.
And yes, brainfuck was there as hyperbole. But I truly believe that there are very few things worse than shell for programming.
$PATH is just an array where each path is an element, so removing or adding a path equals to adding/removing an element from the array which is trivial.
Iterating over files names with spaces works exactly as expected and no IFS tweaking is needed (fun thing, in bash, doing for i in * .foobar; do echo $i; done; in a directory with no files with that extension will return the string "*.foobar"). No "[" craziness, nicer syntax, etc.
Unfortunately fish still lacks other important features (eg. no set -e equivalent).
There is an space for sane Unix shells but everybody has settled on bash and changing the status quo is difficult.
IKR? I perform all administrative operations via a solidity contract, hooked up to a permissioned ethereum chain which my computer scans for relevant logs.
For example:
arr=(a b c)
arr+=(d)
ls "${arr[@]}" # ls "a" "b" "c" "d"
ls "${arr[*]}" # ls "a b c d"
If you do a lot of bash scripting, arrays are invaluable.
#!/bin/bash
command_args=(
--foo bar
--baz "buzz buzz"
)
[[ $frob_option ]] && command_args+=(--frob frab)
echo command_name "${command_args[@]}" "other" "arg"
# Echoes:
# command_name --foo bar --baz "buzz buzz" --frob frab other argIf you're doing a lot of complex calculations, shells are the wrong tool for the job. But if it's a relatively small program whose primary task is invoking other programs on a Unix-like system, shells are still a decent choice. The biggest problems with shells are handled by using shellcheck, so if you're writing shell scripts, use shellcheck.
I use zsh for interactive and bash for scripts for the same reasons as you, though.
Things like array-linked variables ($path is an array version of $PATH and modifications to one propagate to the other), associative arrays (dictionaries in python) and a handful of other really nice tools (e.g. saner white space handling) make going back to bash for scripts unpleasant.
Shell scripting is a different matter. One of the great things about shell scripting is that you can never forget it because you are using it constantly to interact with your system. The fact that it is something you use constantly and can just take and stick in a file and re-use is one reason why shell scripts are so popular.
The problem is trying to put any more functionality into it than that. Everything more complex than this basic functionality should be instead implemented in a script that gets called by make to rebuild the given file.
I just now removed a workaround for that problem from one of my scripts, 17 years after I added it.
https://www.tldp.org/LDP/abs/html/
(Your link, without “www”, gives me a certificate error.)
Support for other “special” characters in filenames (e.g. newlines), however, could still be debatable.
This causes nothing but problems, all to avoid a simple character filter..
https://www.tecmint.com/manage-linux-filenames-with-special-...
See how many "errors" you can find in that article. There are a bunch, consider the touch *, example if your using rm...
this·is·variable·one
one two three
https://en.wikipedia.org/wiki/Soft_hyphen
MargaretAreYouGrievingOverGoldengroveUnleavingLeavesLikeTheThingsOfManYouWithYourFreshThoughtsCareForCanYouAhAsTheHeartGrowsOlderItWillComeToSuchSightsColderByAndByNorSpareASighThoughWorldsOfWanwoodLeafmealLieAndYetYouWillWeepAndKnowWhyNowNoMatterChildTheNameSorrowsSpringsAreTheSameNorMouthHadNoNorMindExpressedWhatHeartHeardOfGhostGuessedItIsTheBlightManWasBornForItIsMargaretYouMournFor.txt
On the contrary! It is filesystems that support plain spaces in filenames that are broken. Filenames are variable names. Allowing separators in them is bonkers. I make a point of carefully crafting my scripts to wreak havoc whenever a user has spaces in their filenames.
There is no particular reason your shell can't distinguish between a space inside a filename and the space between tokens in output. The fact that you have to do anything at all yourself is a bug.
Sure. But there's no reason why typing the spacebar on a GUI to input your filename should produce a file with a plain space on the filesystem. It could be a unicode non-breaking space, for example.
Filenames are the variable names of shell scripting.
If you want to be pedantic, Unix has ALWAYS allowed ANY character in file names, except for "/" (unless you have a GatorBox).
You once said: "Raw spaces in filenames are an abomination that make shell scripting unnecessarily difficult."
No, bash is an abomination that makes shell scripting unnecessarily difficult.
Prohibiting spaces in file names seems to be an obsession with you. (How about tabs? ;) You've said you don't see why people should be able to name their files anything they want, and that your reason for prohibiting billions of people from doing what they want is to make your bash scripts simpler. Then simply don't use bash, instead of trying to change the world. Read what I replied to you when you said that 7 months ago, and tell me if the world has changed so much since then that you're right this time around:
https://news.ycombinator.com/item?id=19952668
Slash is the ONLY character you're not allowed to have under Unix. There are no good reasons to disallow spaces. Disallowing characters in file names that you're allowed to have solves absolutely no problems, it only causes them.
There used to be a bug in the Gatorbox Mac Localtalk-to-Ethernet NFS bridge that could somehow trick Unix into putting slashes into file names via NFS, and Unix would totally shit itself when that happened. That was because Macs at the time (1991 or so) allowed you to use slashes (and spaces of course, but not colons), and of course those silly Mac people, being touchy feely humans instead of hard core nerds, would dare to name files with dates like "My Spreadsheet 01/02/1991".
https://en.wikipedia.org/wiki/GatorBox
I just tried to create a file name on the Mac in Finder with a slash in it, and it actually let me! But Emacs dired says it actually ended up with a ":" in it. So then I tried to create a file name with a colon in it, and Finder said: "Try using a name with fewer characters or with no punctuation marks." Must be backwards compatibility for all those old Mac files with slashes in their name. Go figure!
If you think nobody would ever want to use a space or a slash in a file name, then you should get out more often and talk to real people in the real world. There are more of them than you seem to believe!
I realize that I am preaching in the dark here. This has never deterred me to state my views.
You were wrong and weren't able to support your views seven months ago, and you're still wrong and still can't explain yourself now. In the intervening months have you made any progress in changing even one person's mind not to use spaces in their file names? It would have been a much better use of your time to learn Python or JavaScript in those seven months, since it's bash that's actually causing you problems, not the people who use spaces in file names who you're so compelled to punish.
Back to your argument -- go ahead and state your views: what is your evidence that "Filenames are the variable names of shell scripting"?
Have you submitted pull requests to the Linux kernel and bash and any other affected libraries and applications to eliminate spaces from file names, and have they been accepted yet?
AFAIK shell script doesn't.
> Filenames are the variable names of shell scripting.
Wrong
https://uxplanet.org/the-principle-of-least-astonishment-bc3...
https://medium.com/@donhopkins/the-x-windows-disaster-128d39...
In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex nonsolutions to obsolete nonproblems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry’s standard naked emperor.
Using these toolkits is like trying to make a bookshelf out of mashed potatoes. - Jamie Zawinski
X-Windows: …Even your dog won’t like it.
X-Windows: …The first fully modular software disaster.
Even if you think shell scripting is a good idea (which I don’t), just fix all the obviously dumb toxic stuff like this and put out a shell that is minimally different with only semantic fixes. Emit warnings now for toxic semantics but still support them, and in a couple of years, turn off that support for good.
Make that at least 50 years, if not 100. You have no control over every place where shell scripts are run.
This thread is all about the difference between $* and $@. The difference is explained in the man pages for most shells, but since most programmers don't read instructions, they often need blog posts to explain to them how a documented feature of a language works.
I highly recommend the dash man page (http://man7.org/linux/man-pages/man1/dash.1.html) as a concise explanation of portable shell syntax. From the dash man page, under Special Parameters:
* Expands to the positional parameters, starting from one.
When the expansion occurs within a double-quoted string it
expands to a single field with the value of each parameter
separated by the first character of the IFS variable, or
by a ⟨space⟩ if IFS is unset.
@ Expands to the positional parameters, starting from one.
When the expansion occurs within double-quotes, each posi‐
tional parameter expands as a separate argument. If there
are no positional parameters, the expansion of @ generates
zero arguments, even when @ is double-quoted. What this
basically means, for example, is if $1 is “abc” and $2 is
“def ghi”, then "$@" expands to the two arguments:
"abc" "def ghi"
To answer your question "why still in 2020?", it's because these are independent features people needed. Sometimes people wanted the $* semantics, and sometimes the $@ semantics. So both exist. It's up to you to learn how the system works and use it properly.
It's not like Python doesn't also have weird edge cases that you won't know until you learn the whole language. I've seen people spend hours futzing about with lambdas and list comprehensions to try to fix a bug, which I addressed by just rewriting the expressions as regular-old loops and data structures. Bash isn't uniquely bad, it has warts like everything else. Take out the warts you don't want and someone else will complain that they're missing.
bfox (wrote bash): Nah, Bash disproves that.
A colleague answered me: "If we left, there's around 500 people in this building who could support my Bash script, and around 50 Python people."
It kind of stuck with me. At this point, I feel like Bash is one of the common tongues between all tech roles (that deal with Linux, that is).
Python, while nice, is a lot more niche, since you really have to be into development to know Python, while every sysadmin worth their salt can debug a Bash script. And, of course, every SWE, TSE, DOE, .... worth their salt know Bash scripts as well.
If you want to get a job (in the Linux land), bash scripting is typically an "of course".
On the other hand most people don't write bash scripts and usually have to deal with them when encountering legacy systems or languages.
That's not a good sign.
They were both totally valid; we just hadn't agreed on a calling convention for the tool so people were trying to fix their individual problems based on their own habits.
You almost always want "$@".
If you're using $* , you're not supporting arguments that contain spaces, and you'll break if arguments contain wildcards. If you're using "$*" , you're treating multiple arguments as a single argument.
There is never a legitimate reason to use $* . Even in the rare cases where you want those semantics (hint: you don't), you should use something like "$(join ' ' "$@")" instead.
> They were both totally valid;
No, they weren't.
> Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
The trouble with `communicate()` is it only handles very simple cases, basically, your output has to fit in memory. Same problem exists in asyncio.[2]
Yes, you can usually work around this. That doesn't mean it's a good replacement; whereas complex pipelines are so trivial in bash that any user-defined function can be used in a pipeline, subprocess generally forces you to do the dumb thing and create a mess of temporary files.
And that's not even considering you're writing 10 times as much code than you would to accomplish the same task.
[1]: https://docs.python.org/3/library/subprocess.html#subprocess...
[2]: https://docs.python.org/3/library/asyncio-subprocess.html#as...
I think in part it's because the POSIX-style shell is so tightly wound with the OS, it's extremely proficient at spawning and forking processes, working with files, and communicating, and that experience is seamless. Python feels like a different world, and doing any subproc, pipes, or file i/o always feels like crossing some boundary and back.
This is actually one of the main advantages of using Python. You can solve all those deadlock issues through careful use of threads, which are much better than the Bash sub-shell equivalent.
If anyone has an example script that requires the use of those functions and is cumbersome in Python, please share it. We can always improve the Python standard library if necessary.
Plumbum is cool, but now I have an extra [""] for every command, and I need python and a lib (which usually means an env as well) to bring an app up.
On the other hand, I try to migrate as much business logic into the app as possible, and increasingly kv as well, so bash is pretty much pointing at config files and commands.
// echo_and_run() { echo "$*" ; "$@" ; }
Logs the cmd before running.
https://www.gnu.org/software/bash/manual/html_node/The-Set-B...
e.g. my answer https://unix.stackexchange.com/a/41595/3169
foo.sh --clean bar.yml
And actually run something like:
blah -e '{"clean": true}' bar.yml
where -e and the thing in '' are two separate args...
Then I tried it. Either I'm an old dog or I was naive.
Both are true, I think.
echo dollar-star: for i in $; do echo $i; done echo dollar-at: for i in $@; do echo $i; done echo quoted-dollar-star: for i in "$"; do echo $i; done echo quoted-dollar-at: for i in "$@"; do echo $i; done
./x.sh a "b c" d dollar-star: a b c d dollar-at: a b c d quoted-dollar-star: a b c d quoted-dollar-at: a b c d
Blank lines separate paragraphs. Text surrounded by asterisks is italicized, if the character after the first asterisk isn't whitespace.
Text after a blank line that is indented by two or more spaces is reproduced verbatim. (This is intended for code.)
Urls become links, except in the text field of a submission.