Shell-ish scripting in Go with ease
github.com
github.com
Why shouldn't it be as easy to write system administration programs in Go as it is in a typical shell?
2. Shell scripting is a combination of a grammar, operators, a few simple functions, and an extremely loose coupling with generic i/o and logic. I don't know Go well, but it probably doesn't support a similar flexibility. (most languages are very proscriptive about how you can use the language, so you usually can't make things as easy as they are in a different, more tailored language/interface/paradigm. this is why we have DSLs)
3. Programmers don't really understand the concept of productivity [outside of programming itself]. A programmer would solve a problem by taking 6 weeks to design a perfect program to do the thing. A Sysadmin would take 5 minutes with a shitty language and a shitty tool and get way more done in less time. And re-writing everything into a Go library would always be slower than shell scripting, because it requires re-implementing what a shell script would just use as-is.
Scripting is duct-taping the wheel rather than reinventing it. If you want to save yourself a whole lot of time and trouble, just use the duct tape.
(also: don't go templates exist? why isn't that used for scripting)
Most people aren't sysadmins, but occasionally have to do sysadmin-like things. I've been programming with python and go for years. I've never been able to get the "core" command line utilities to really stick in my head. A sysadmin uses them every day, whereas I rarely have to reach for them. On the rare occasion when I _do_ have to reach for them, it is excruciating (what was the flag I need for `find` again?). If it were life or death and I had to debug even the simplest sed/awk command, it would be death for me! But this package makes perfect sense to me and really enables me to write this sort of quick and dirty thing in a language I'm familiar with and can confidently maintain.
This isn't for everyone, but there's definitely a population that can get a lot of value out of this.
This is not on you, `find`'s command line syntax is awful even if you have to use it everyday.
What's awful about it?
That alone is why I prefer `fd` or plocate for most tasks I previously used find for.
> [It doesn't] follow the convention of many other CLI utilities.
Because unlike other CLI utilities, find evaluates an expression specified as arguments. Maybe a small DSL like JQ would be better, but how would you embed shell code in it? Meh.
However for the vast majority of queries, I'm only looking for something based on the filename, and that's where having sane regex and standard argument structure (like fd has) is great.
I'd manage with find|grep but yes, nothing wrong with wanting something more compact.
> sane regex
For me that's POSIX ERE, I don't know which flavor fd uses.
That, combined with reading the man pages if it doesn’t work first try has been really effective for me.
Let's say you're an "Engineer" engineer, and you deal with units of measurement. You grow up in the US, so you first learn the Imperial system. But so much of the rest of the world uses Metric measurements. Do you find and acquire Imperial versions of every tool, fastener, etc, because it's what you're familiar with? Or do you learn Metric as well, so you can access all of the tools, fasteners, etc found all over the world?
Or take an example from a thousand years ago. Say you're a trader in the Mediterranean and you want to sell your wares. Do you only sell in your local town, where everyone speaks your dialect? Or do you pick up the "Frankish Language", the pidgin language spoken by sailors and other Western Europeans? Learning this mix of Venetian, Catalan, Portuguese, Tamazight, Turkish, Greek, and Arabic will give you extra skills you can use to trade with a vast array of peoples.
POSIX is the closest we have to a 'Mediterranean Basin' for operating systems, and shell scripting is its pidgin language. You don't have to learn it to make a living, but it sure as hell helps.
If you are making a genuine effort to avoid the shell, you won't ever learn these basic flags -- you are holding yourself back. It is like a person complaining about using the stairs -- "On the rare occasion when I _do_ take the stairs, it is excruciating..." -- the problem is not necessarily with the stairs.
[1]: https://leancrew.com/all-this/2011/12/more-shell-less-egg/
The major reason shell scripting is nice is because its portable with a copy/paste and HelloWorld.sh doesn't need a compiler or a VM or even any preceding incantation to run.
As much as I hate that this is where my hopes are, I do hope that Powershell gets to a point where you can write single C# files and run them.
Together with intuitive function calling facility using Uniform Function Call Syntax or UFCS you can easily has natively compiled scripting environment [2],[3].
[1] Pragmatic D Tutorial:
https://qznc.github.io/d-tut/hello.html
[2] Why I use the D programming language for scripting (2021) (50 comments):
https://news.ycombinator.com/item?id=36928485
[3] Uniform Function Call Syntax:
Tcc has the `-run` flag for easily doing this with a normal-ish shebang.
With a nasty polyglot preamble of C and bash at the top of your file, you can do it with any compiler.
But if you insist, D now supports and can compile C language that you can perform using rdmd [2].
[1] C shell:
https://en.wikipedia.org/wiki/C_shell
[2] Adding ANSI C11 C compiler to D so it can import and compile C files directly (105 comments):
I know Go okay-ish, and you can remove the 'probably' from that sentence.
Hell, even using Python for shell scripting is a pain in the rear, and that's way more flexible than Go.
The author is trying to bring the better type checking, concurrency, deterministic syntax and error handling of go to the world of shell scripting.
Most scripts end up becoming programs themselves and go beyond their original scope. It would be nice to force better concurrency & error handling early on before the script takes on 5k lines of functionality.
It was my first time writing a golang project at work, so I'm sure it could have been better. But writing it the naive way, with all the required golang error handling, it ended up taking about 10x more lines of code in golang than the original bash script.
It does have a dramatically better UX (largely thanks to spf13's cobra and viper), and is way faster than the original, and the codebase is a lot cleaner and more maintainable. So I think it was worthwhile for the users and maintainers.
But still, 10x more lines of code. I like the OP, but I'm still not sure I would reach for golang for short shell scripts.
I'd probably end up rewriting it as 50 lines of code that used a bunch of pure perl libraries and then use https://p3rl.org/App::FatPacker to bolt the dependencies onto the front for distribution so it was still a single file to install for everybody else.
(there's a lot of perl out there that uses techniques I would switch away from as soon as I got past a one liner in the middle of a pipeline, alas, but it doesn't have to be that way, perl just doesn't stop you blowing both feet off ;)
Posix shell-compatible scripts will also likely work on all platforms where you go program would've been run.
While I see your point, writing a Posix compatible shell script is not trivial. Little errors creep in that "work on my machine" because /bin/sh is a symlink to /bin/bash, but break the script when someone runs it on macOS.
In my experience, you get a lot of cross-platform compatibility when writing Go for zero effort.
To sum it up, the biggest complaints are error handling, null handling, and dependency management. And y’know, being backed by a company of ghouls hellbent on extracting value for themselves at the expense of society.
I don’t personally have a bone to pick with Go mod, save for how the GOPROXY DoS issue was handled.
go mod is the second best I've used for sure, but if someone releaed bundler-but-for-go I'd switch to it in a heartbeat.
Google is an exploitative monopoly. There’s been plenty of ink spilled on the subject to the point that I feel no obligation to repeat it.
While it’s true that GVR is currently employed by Microsoft, the ecosystem of Python is far more anarchic and decentralized.
i.e. Whether a language exists "at the expense of society" probably depends less on who makes the language, and more on what you do with it.
Architecturally, shell scripts should _exclusively_ be for bootstraps, configs, or extremely localized (individual developer) automation.
The New York Minute you need non-trivial error-handling/flow-control it's no longer a "shell script" and deserves a proper rewrite in a proper programming language.
Ian Malcom's quote from Jurassic Park comes to mind:
"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should."
If you look at the Python build-in API, it's pretty terrible. In all fairness, Python strives to work equally well on Windows, Linux, and some other things. Good luck with that.
I prefer doing systems scripting in Python over Bash. But writing converting a Bash script to Python without "Pythonizing" it is a bad time.