Hacker News Reader

Comment by rob74 | Hacker News Reader

This may be controversial, but sometimes I feel the biggest mistake is using a shell script in the first place. For example, on servers where you have PHP installed anyway, a PHP CLI script is often an alternative. Say what you will about PHP, but PHP code is much more readable than shell scripts - especially if everyone working on the server is fluent in PHP anyway. Shell scripts do have their niche, but often you find that the scope of a script keeps growing, then soon it becomes cumbersome and makes you wish you would have chosen another alternative from the start...

View on HN

Once I get a script to a point of defining functions or anything but the simplest if or for loop, I rewrite in a proper language (Perl, Python, even PHP) and bundle it as a deb onto our internal repo.

Too many times I didn’t move from bash until too late.

I still write bash, I’ve got one which runs up a 20 line ffmpeg command with a couple of variables, that’s not a problem. On the other hand I changed the complexity of a file analysis tool from grep/sort/uniq/sed to Perl a couple of weeks ago before it became too large.

Not controversial at all!

Most linux distributions come with python installed. Anything more than invoking a binary and redirecting the output? Just write it in (pure) python I say!

Edit: by pure Python, I mean don’t require any `pip install`s

Python is terrible at juggling files, setting up pipelines and working with processes. The resulting code is more complicated than the equivalent bash, not to mention longer, and hidden gotchas abound (subprocess deadlocks anyone?). I'll take 50 lines of bash instead of 200+ lines of Python.

How does a subprocess deadlock happen in Python? Why would this be specific to Python, and not caused by deadlock in general?

I agree that 50 lines of bash is generally more maintainable than 200 lines of Python.

This has only been true for the smallest shell scripts in my experience: bash will be less code at first but once you to be portable, handle errors, locking, unicode, buffer or process output, perform non-trivial redirections, or need any data structure more complicated than a simple string it's been pretty common to replace hundreds of lines of spaghetti bash with less Python even after you add comments and a proper CLI interface.

I've never had subprocess deadlock — was this on Windows?

> I've never had subprocess deadlock — was this on Windows?

I've seen this happen seemingly randomly in linux trying to pipe the stdout of one subprocess into the stdin of another.

Interesting, I've never had that happen in heavy usage. I've had cases where the called processes blocked for some reason but that was solved by adding a timeout.

Welcome to (shameless plug) Next Generation Shell. It's a modern programming language for DevOps. It is both concise for running external programs (uses small subset of bash syntax) and it is a "real" programming language with data structures and sane error handling among other things.

Compared to general purpose programming languages, it was built for "DevOps"y use cases in mind. See https://github.com/ngs-lang/ngs/wiki/Use-Cases

Ability to run external programs is high on the priorities list. In NGS it means having its own short syntax and handling exit codes among other things. NGS throws exceptions on "bad" exit codes. Hint: not for all tools non zero exit code is "bad". Did not see equivalent exit code handling anywhere else.

Compared to bash... It's not fair comparison even. Another era, another reality, other expectations. Small example: When APIs return structured data, well... you better be able to handle it. See https://ilya-sher.org/2018/09/10/jq-is-a-symptom/

Compared to other modern shells, I would say, NGS is programming language first as opposed to shell first.

I once worked in an R shop where a lot of “system scripts” were written in R instead of shell.

It made sense: everyone could read R, not many could read bash/shell as most had learned R on a windows PC in grad school of statistics.

So while sometimes a bit clumsy and not very portable, these scripts could be read by anyone.

This is really huge. If a script is going to be used by anyone other than yourself, it needs to be readily accessible (e.g. understood) for the group that will be using it.

Agreed! If it’s more than a few lines I switch to something else usually Python these days.

Couple decades ago even wrote some init scripts in Perl because they needed to be complicated.

Why not use Ansible?

Ansible didn't exist 20 years ago.

The question is "why not switch to Ansible instead of Python" rather than "why didn't OP use Ansible 20 years ago instead of Perl".

Seeing that Ansible is basically a set of Python libraries that enable users to write declarative (and hopefully idempotent) scripts, it isn't a stupid question.

hmm, I can take a stab at that. I have extensively used both. Ansible is pretty good for 'get something or set of machines into a particular state'. Python is pretty good for 'run these bits and give a nice error if something goes weird'. The 'error path' is where ansible falls down for me. It can generate some very strange ones. Also its yml syntax can sometimes create odd issues with people who have not used it before. Ansible is also terrible for any sort of data processing. It is very good at iterate over a list and do this action to those items though. I recently did an audit script using ansible. Because I thought hey I have this cool tool to do it. It was a mistake. It probably would have taken 1/3rd the amount of time to do in python. Also if someone (probably me) has to extend that script getting the yml right will be tricky. I also recently used ansible to write a bunch of upgrade scripts. It was a breeze and would have taken much longer to do in python. In my case many times you have to write to the lowest common denominator. So bash it is, with some sort of central ansible setup controller.

I think the YAML syntax is definitely one of the weakest points of Ansible user experience, especially if you're creating complex books/roles. It's extremely long, and when you try to do anything slightly more complex in the YAML syntax itself, e.g. handle an error of a module, it gets very hairy.

As a side note, Bash + Ansible tends not to be an amazing combination, unless you can write idempotent Bash scripts, IMHO. That sounds like a recipe for false safety, where you think you wrote strong automation, and someone that is used to pure Ansible will quickly realize that it does not work in specific conditions, e.g. after second time or after specific conditions are run multiple times in a row.

I agree about the bash+ansible bit. For me it is usually using ansible to push bash scripts to some remote machine. Those remote machines may or may not have python or ansible. So ... bash for those bits :(

Shell script is a kind of DSL for the OS operations and PHP a kind of DSL for the web programming. Similarly, it does not make sense to advocate the usage of R for web programming but people do it anyway.

Shell script have their deficiencies but it's the best for their intended environment. Alternatively now there's Oil shell that looks promising for modern approach to shell programming [1]. Expect to see exponential usages of shell programming now that we have new popular OS related technology including containers, isolates, and functional package management like Nix and Guix.

[1]https://www.oilshell.org/

Use whatever tool works best for you. I use node.js for "shell scripting" on my own systems because that's what I use anyway for other stuff.

The advantage I find with Python is that it's commonly part of the Linux systems I come across (e.g. Debian-based and RHEL-based Linux systems). I'd have to install the Node.js runtime. So despite preferring Node, I find myself writing more and more Python.

Reply on news.ycombinator.com