Hacker News Reader

Comment by moondev | Hacker News Reader

I used to lean on python for much of my scripting needs, mainly because the more advanced bash syntax was pretty daunting. Getting better at bash has a trickle-down effect, especially in this container age. ENV var scoping + loops and various var expansion methods really made it click for me. Shelling out to various tasks (and grabbing the results) is effortless via bash scripts. With bash on windows now it's pretty much ubiquitous. My advice is to consider why the task at hand can't be done in bash, because often times it can, with much more portability.

View on HN

In both my personal and professional life, I have the opposite conclusion. Bash is only for the simplest scripts and pipelines, and everything else (assuming you're going to use it more than once) gets written in Python or Go.

The Bash syntax is not daunting, or if it is that's never been the problem with Bash. The problem is that Bash or shell programming in general gives you a million ways to shoot yourself in the foot and maybe one or two obscure, funny-looking ways to do what you want. Like iterating over the files in a directory, for example. If you think that's easy in Bash you have either have a funny definition of easy or you're forgetting a few corner cases. Or getting the output from a program—did you know that $(my_prog) or `my_prog` modifies the program output?

For containers we do everything declaratively.

Fair enough! I'm more in the devops/platform space so worrying about python versions and setting GOPATH is wasted time for my purposes. What I love is the portability with minimal effort.

I'm in a similar position. Unfortunately I've seen far too many non-portable Bash scripts fail to run in production and I can't trust them! This includes simple errors like #!/bin/sh at the top, missing programs, or differences between GNU and BSD versions of programs (macOS and Linux). Chalk it up to differences between dev workstations and production. Python has some problems too but they tend to be pretty well-understood and navigated by our developers.

GOPATH isn't actually necessary except when you compile, and Python version problems mostly come up with larger programs you'd never dream of writing in Bash. 2.7 is dirt common even if you're using a slow-as-molasses-update-cycle LTS Linux.

> Like iterating over the files in a directory, for example. If you think that's easy in Bash you have either have a funny definition of easy

Maybe I'm overlooking something...but why wouldn't this work:

for file in $(ls); do {<looped command>}; done

  $ mkdir "hi there"
  $ mkdir "how are you"
  $ ls -l
  total 8
  drwxr-xr-x 2 xxxxxxxx xxxx 4096 Jun 26 14:54 hi there
  drwxr-xr-x 2 xxxxxxxx xxxx 4096 Jun 26 14:54 how are you
  $ for f in $(ls); do echo $f; done
  hi
  there
  how
  are
  you

ls | while read file; do echo "$file"; done

... works better for the easy edge cases, but still probably has some issues. Personally I think klodolph called it; once you get into anything that has a few interesting edge cases bash becomes pretty unwieldy.

The easiest way is still: for file in *; do ...; done

This works fine with spaces and other strange characters, no need to come up with more complicated ways.

If you must use find, the correct way is really tricky and uses obscure bash features like settings IFS to the empty string to split on null bytes and process substitution so that the loop doesn't run in a subshell (which would prevent it from modifying variables in the current environment):

while IFS= read -r -d '' file; do ... done < <(find /tmp -type f -print0)

See http://mywiki.wooledge.org/BashFAQ/020

I like that, I've never thought of or seen it, although it should've been intuitively obvious given how bash expansion works.

I think this demonstrates the point pretty well though - it takes a discussion among three of us before arriving at something that works moderately well, and we found four fairly different ways of doing it...

You can use

  find . -type d

or set IFS.

  IFS='
  '

  or

  IFS='\n'

then reset it later. Perhaps store existing IFS in a variable so you can put it back. The \n vs actual line feed has to do with an edge case in cygwin and dos files.

Fair enough.

for file in *; do { ... }; done

is both much easier and much safer than piping the output of ls.

It does the correct thing with filenames containing spaces or weird characters, and the intent is much more visible, so there is really no reason to use $(ls).

More details: http://mywiki.wooledge.org/ParsingLs

Reply on news.ycombinator.com