Even then, my threshold for “this should be Python” has shrunk over the years: I used to say “greater than one screen of code” but now it’s more like “>1 branch point or any non-trivial scalar variable”.
It's still better to script in Python or Ruby than Bash. Nobody understands Bash. It's even more mysterious than Perl.
1. more emphasis traditional programming paradigms (be it JS, LISP, Python, whatever) which leaves a platform that is arguably a better designed language but however is a poorer REPL environment for it. Bash works because it's terse and terseness is actually preferable for "write many, read once" style environments like an interactive command prompt.
2. or they spend so much effort supporting POSIX/Bash -- including their warts -- that they end up poorer scripting languages.
I think what we really need isn't to rewrite all our shell scripts in Python but rather better shells. Ones which work with existing muscle memory but isn't afraid to break compatibility for the sake of eliminating a few footguns. Shells that can straddle both the aforementioned objectives without sacrificing the other. But there doesn't seem to be many people trying this (I can only think of a couple off hand).
Not that I'm taking anything away from zsh. It is a nice shell. But I think we can do even better considering how dependant we still are on shells for day to day stuff.
I agree with the idea of breaking backwards compatibility, but Powershell honestly has enough core design issues that it itself is starting to feel like it needs a major backwards-incompatible update.
Powershell 7 has a full Win32.
I mean, I would expect the syntax to be something like:
make >/dev/null 2| more
Note that the order of redirections
is significant. For example, the
command
ls > dirlist 2>&1
directs both standard output and
standard error to the file dirlist,
while the command
ls 2>&1 > dirlist
directs only the standard output to
file dirlist, because the standard
error was duplicated from the
standard output before the standard
output was redirected to dirlist.
Can it be memorized? Yes, because it is just a single 'incantation': "2>&1". Just put that redirection operator before the redirection of the standard output to the file, and the result is stdout goes to the file, stderr goes to the pipe.
"Redirecting streams" winds up being confusing - it's all just `dup2`.
Note that the order of redirections is significant. For example, the command
ls > dirlist 2>&1
directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command
ls 2>&1 > dirlist
directs only the standard output to file dirlist, because the standard error was made a copy of the standard output before the standard output was redirected to dirlist.
Yes, I know that underneath it's all calls to `pipe()` and `dup2()`, which I can do (and have done) in a language other than shell. It's the shell redirection syntax (for anything more complex than simple redirection or a pipe) that just doesn't make sense to me.
command 2>&1 > >(cat - | cat - > fd1) | cat - | cat - > fd2
How to read this:
When Bash sets up the file descriptors for 'command' it initially makes descriptor 1 refer to the pipe, and descriptor 2 refer to the terminal.
So, the dup operator (2>&1) copies the fd in descriptor 1 (which is the pipe) into descriptor 2 (so after the dup operator is processed both stderr and stdout for command reference the pipe).
Next descriptor 1 is replaced (the > operator) by a reference to a fifo created by the process substitution operator (the >(...) operator).
So now command's descriptor 1 refers to the fifo created by >() (which itself contains a "string of commands").
Then, because stderr for command was made to be a copy of what was previously stdout (before modifying stdout) it continues to refer to the pipe Bash setup (the | operator), and stderr now flows out over the pipes to another "string of commands".
( echo "stdout"; echo "stderr" 1>&2 ) 2>&1 1>/dev/nullBest of all though, it's absolutely compatible wherever you need to run it.
"If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date."
Cuz I've never heard it until this thread, where it pops up multiple times. Great phrase but went 0-60 in usage.
This is true, especially of Python scripts.
Edit: Getting some downvotes, so to clarify, I am indeed saying that concerting something from Python to Perl has a high likelihood of making it for less maintainable. I get that people can write good perl. As someone who has had to maintain perl in the past; the fact is that it's far more common for the end result to be horrible perl. I have some issues with Python, but it is FAR more maintainable than perl.
Every single person who doesn't use Perl every day, professionally knows a different subset of Perl. This is one of the few times where being more knowledgeable than the interviewer is also a problem. You will write something, and the interviewer will question you on it because he has never seen it.
I finally solved this problem by bringing one of my personal programs written in Perl and also rewritten in Python so we could talk about it. Now, the interviewer is in MY subset of Perl AND can't argue because I have working Perl code in front of him. This was back in the 1990's when everybody in VLSI design expected you to know Perl.
Because of this "different subset" issue, if you want to maintain a Perl script, you have to basically know the entire language. This is what makes maintaining Perl scripts so difficult.
This "different subset" problem is the whole reason I left the Perl ecosystem back in 1996(!) at the height of Perl's popularity and never looked back.
Perl5 still works fine, and is maintained.
> Is it a worthwhile replacement or too little too late?
False dichotomy - see above.
I have actually found that is pretty much always a problem. If you pull out something the interviewer is unfamiliar with they will often assume you are full of it. I have had such people refuse perfectly good explanations for things they haven't heard of because they assume incompetence before that possibility.
I'd take those experiences as a blessing rather than a curse.
Last I knew, that position was still open almost a year after the fact.
Has this changed at all?
While I still regard myself as a vastly better VLSI designer than programmer, my ability to wrangle software the whole way from assembly language on a chip to just shy of the top of a full web stack pays far better than my ability to wrangle transistors. And, in my opinion, attacks far more interesting problems.
At work lately, there's been a spate of contorted "just why?" Python scripts that could've been accomplished elegantly in a handful of lines of shell. While no one would select shell languages as the ideal for a lot of complex logic, data parsing, etc., there's no competition for a shell when you need to do what shells are meant to do: chaining invocations, gluing and piping output, and so on.
I'm not sure about that. Google certainly has some very talented people among their developers, but most of them are so bad at programming that they had to invent their own heavily watered down programming language.
That's where we came in with the need to use "$@" vs the unsafe-by-default bare $@.
There is no reason this shouldn't be easy to do in Python too, if it is then it is something a library should fix.
This, I say as someone who loves Python and I use it as my primary language both privately and at work. But I have to admit, Python scripts do not really age well.
Before that, well it's been a while. But with and yield were added as a keywords (breaking any code using them as a variables). The xreadlines module disappeared at some point (I'm sure a few other modules have died along the way). At some point you couldn't raise anything you wanted as exceptions any more. Oh, and the source encoding declaration. That's what I remember off the top of my head.
Minor things that are handled in things that are maintained, sure. But many things are also built once and then expected to run for a long time - especially in the role shell scripts usually fill.
Note, that I don't argue that any of these changes were bad. They were all good changes that made the language better. But, they resulted in old code breaking. One extra headache to deal with for the poor schmuck that was responsible for upgrading this server or that to the latest OS release.
I'll provide the counterexample.
Any scripts I wrote more than 10 years ago for actual "Bourne shell" /bin/sh often fail in mysterious (and sometimes silent) ways when run under bash.
Bash is just as guilty of backward incompatibility as Python is. Python at least generally has the decency to squawk about it.
Over the years I've spent way more time going into inconsistencies about the various platform tools (e.g. GNU vs. BSD implementations of sed, grep, find, etc. even before you ge to the space aliens-with-broken-translators realm of AIX) — which using Python avoided needing to care about — or library / file naming changes. Similarly, a few years back there was way more time used when various HTTPS improvements flushed out code using old versions of OpenSSL or, worse, gnutls.
https://www.python.org/dev/peps/pep-0466/
(Can't remember the details, but it may have been something about some extra hoop you'd have to jump through to trust self signed certs that broke our testing infrastructure)
I have done something similar too once in a shell-like Python CLI I worked on. Can also use __ror__ to be able to pass primitives into your commands like `[1, 2, 3] | SumCmd() | PrintCmd()`
I do agree that typing something like that, especially when working in a shell, is much nicer than having to go back and forth all the time to type `print(sum([1,2,3]))`
Here it is mentioned in a talk about Python Aesthetics by Brandon Rhodes: https://www.youtube.com/watch?v=x-kB2o8sd5c&t=8m24s
I'll stick to Python, but Ruby is worth a look.
It just challenged my colleague to make the code denser, to stay within the given limit. Sigh.
Although I think all of these things are just complicated ways of saying "please seriously reconsider writing it in bash."
But I write bash scripts all the time, I just try to keep them as short and simple as possible.
My point here is it is less to due with the language and more to due with the mindset when solving a problem.
The main issue I see with more inexperienced devs with bash is that they tend to think it’s okay to be lazy with the code because it’s just “bash”. If you would write safety checks and comments in your python you should be doing the same in bash really.
shellcheck /scriptsdir/script
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang.
shellcheck -shell=bash script
Unknown shell: hell=bash
Although there may be some irony in the fact it cannot sort out it own argument parsing.
1. I do not use other scripting languages such as Python, Perl, Ruby, etc. That means, e.g., for quick and dirty one-offs and prototyping, I can omit the shebang. Debian's "dash" scripting shell is derived from NetBSD's ash, the one I choose for interactive use.
SC2148: Tips depend on target shell and yours is unknown. Add a shebang.
I always have the top of my shell scripts with a shebang, even if the script isn't intended to be directly executed.
Pick the user's bash from PATH environment:
#!/usr/bin/env bash
#!/bin/bash
#!/bin/sh
#!/usr/bin/env python3
#!/usr/bin/env vi