I recently made a dashboard to compare four versions of awk output together, since not all awk scripts I'll run the same on each version: https://megamansec.github.io/awk-compare/ I'll have to add those:)
It runs an action for each line in the input (optionally filtered by regex). You get automatic variables $1,$2... for the words in the line split by spaces.
The syntax is almost like a simple subset of Javascript. Builtin functions are similar to C standard library.
If you have input in text that is separated in columns with a delimiter, and you want do simple operations on that (filter, map, aggregate), it can be done quickly with awk.
That's all you need to know about awk.
I'd argue that it should be every programmer who doesn't already know a scripting language like Ruby or Python. If you already know a scripting language, chances are the time saved between writing an Awk one-liner and quickly banging out a script in your preferred language is negligible. And if you ever need to revisit it and maybe expand it, it'll be much easier to do in your preferred scripting language than in Awk, especially the more complex it gets.
I'm speaking from experience on this last point... At my work I wrote a very simple file transformer (move this column to here, only show lines where this other column is greater than X, etc etc) in Awk many years ago. It was quick and easy and did what it needed to. It was a little too big to be reasonable as a one-liner, though not by very much at all. But as we changed and expanded what it needed to do, it ended up getting to be a few thousand lines of Awk, and that was a nightmare. One day I got so fed up with it that I rewrote it all in Ruby in my free time and that's how it's been ever since, and it's soooo much better that way. Could have saved myself a lot of trouble if it were that way from the beginning, but I had no idea at that time it would grow beyond the practically-a-one-liner size, so I thought Awk would be a great choice.
There are lots of things "every <tech position> should learn", usually by people who already did so. I still have a bunch of AI/ML items on that list too.
What's the advantage of learning AWK over Perl?
- Awk is on more systems than Perl
- Awk has more implementations than Perl
Asking a new hire to "learn awk" vs "learn perl" have two very different time investments attached to them.
Tasking someone with "learning a subset of perl" begets the question "what subset?", and a very exhausting conversation with someone(s) routinely asking "so?" follows. After spending a large amount of time re-litigating which subsets of perl features we want that awk already supplies.
so?
> Awk is on more systems than Perl
By what metric?
> Awk has more implementations than Perl
so?
Awk is older and as a part of POSIX the version found on unix-like environments will be (outside of extensions) compatible with others. If one or one without the extensions you want isn't present you can choose an implementation, even one in Go and it'll work.
Perl, and I've been writing Perl since Perl4, doesn't have those characteristics. It's a much more powerful language that has changed over the years and it is not always present by default on a unix-like system. Because the maintainers value backward compatibility, even scripts written on Perl5.005 have a fair chance of working on a modern version but it's not assumed (and you shouldn't assume anything about modules). Because Awk is fossilized, you can assume that.
One reason for this is that the popular BusyBox project includes an Awk implementation: BusyBox Awk.
Pretty much everywhere there is BusyBox, there is an Awk, unless someone went out of their way to compile it out of the BuxyBox binary.
Perl is more regular than Awk for the simple cases and is more usable for anything that isn't merely iterating over input.
Of course, you shouldn't any of awk/perl/shell for tasks that aren't being run by you or are over say 20 lines long.
I reach for awk when my bash-scripts get a bit messy, perl is/was for when I want to build a small application (or nowdays python).
But both perl and python require cpan/pip to get the most out of and with awk, I just nead awk.
Because you (or someone else) can run your Python later if needed, and have confidence the output will be the same.
Sure, there are some times when a one-liner is needed, and you can always put that one line in a document for others to run. I can think of many times when I was on-call and needing to grep some data out of logs that wasn't already in a graph/dashboard somewhere. When time is of the essence, or if you're really really sure that you won't need to run the same or similar thing ever again, even if the data changes. I even changed my shell to make up-arrow go through commands with the same prefix instead of linearly traversing command history because I had so many useful one-liners I re-ran later.
But as I've gotten more experienced, I've come to appreciate the value of committing those one liners to code as early as possible, and having that code reviewed. Sometimes a really useful tool will even emerge from that.
It also doesn’t let you reorder or splice together fields.
I used it for years but now that I have a working understanding of `awk` I have never looked back.
Of course awk can do much more, but if all you want is "| awk '{print $2}'" then "cut -wf2" is so much more convenient.
Getting awk in your head (fully) takes about an afternoon: reading the (small and exhaustive) man page, going through a few examples, trying to build a few toys with it. Perl requires much, much more effort.
Great gain/investment ratio.