Hacker News Reader

Comment by e63f67dd-065b | Hacker News Reader

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

View on HN

If you're in perl all the time that probably makes a lot of sense. For me, awk is one of the few languages that I can safely set aside for months and then I'm back up to speed in 10 minutes. There's just something very intuitive about it, and it somehow fits very naturally with other common command line tools.

Weird, I feel the same way about Perl.

If you are comparing Awk vs Perl for scripts, I'd prefer Perl (or Python).

This post is about short one-liners for ad hoc use cases. I prefer sed/awk over Perl for such cases. Though, if you already know Perl, you could continue using it instead of having to learn more tools.

Do all systems still come with Perl baked in these days? If so I could see reaching for that over awk/sed. If I have to install a runtime I may as well just reach for Python

A bunch of the default git extensions are written with Perl, so you will find some version of it available on most modern Linux systems

>> Do all systems still come with Perl baked in these days?

If you use Git for Windows (https://gitforwindows.org/), it includes Perl.

…and gawk :)

Yes. Frequently any tool set that has gawk will also include sed, perl, cut, head, tail, less, vi / vim, etc.

It is nice that Git for Windows includes bash and all these tools.

What are "all systems"? Most mainstream Debian or Fedora based systems install Perl by default (but not necessarily in specialized settings such as embedded/boot/rescue systems). Alpine linux does not include it in standard images. FreeBSD (and probably Net/OpenBSD) don't install Perl by default. The current macOS still includes it, but Apple has notified that it will be removed at some point. Windows does not include Perl or awk by default.

Sed and Awk are part of POSIX, and maybe more importantly also part of Busybox. They're almost always available when Perl is available, while the reverse is not true.

Awks super power, and the reason I mostly use it, is it's free read loop, free field splitting, and the pattern/condition matching model.

As a LANGUAGE, it's "eh". It just happens to be "good enough".

You can, of course, do all of that with Perl. But then I have to write all that boiler plate I get with awk for free. And the gains in Perls language aren't enough, for me, to dump awk. And I don't use it for "scripting", I use it for data processing, tearing up files for mostly one off tasks. So I don't miss Perls depth. If I want depth, I'll go somewhere else.

> free read loop

perl -n

> free field splitting

perl -a ... $F[1/2/3/etc]

> pattern/condition matching model

Not quite sure what you mean, but `perl -lane 'print if /abc/'` is might be what you're looking for

The boilerplate can be mostly eliminated with the magic incantation of `perl -lane`. The trick that makes all this work is that perl defines a whole bunch of pre-defined variables and populates them with things that might be helpful (see $_, @F, etc).

There aren't any technical advantages, no. Perl's features are a proper superset of awk (by design!).

What's happened is that Kids Today (tm) never learned perl. So they're discovering awk as someone new to the idea of stream processing. And awk was a great idea for that, and it represented a genuine innovation worth emulating.

In the late 1970's. Then of course perl did emulate and surpass it. But then got forgotten. So kids are discovering awk instead. It's a little cringe, really.

For simple uses cases, I find awk simpler than Perl. I love Perl, have written tens of thousands of lines, but on the CLI I prefer awk. I’m sorry I “cringe” you.

Awk is a useful language that you can learn in one afternoon, after reading the man page and a few examples. And then you can spend your whole life using it for several projects. You cannot do that with perl. That's why awk has a longer shelf life than perl.

> And then you can spend your whole life using it for several projects. You cannot do that with perl. That's why awk has a longer shelf life than perl.

A Perl developer would of course say you have this completely backwards and even if I haven't programmed Perl much, or even at all for the last decade I would tend to agree.

Why is that cringe? They genuinely probably came across awk before perl (I know I did, I read "The AWK Programming Language" and then went on to "The C Programming Language"). Having that said, awk is great and it's been the same for decades and available on every system (the same can't really be said about perl).

>> awk is great and it's been the same for decades and available on every system (the same can't really be said about perl).

The only issue with AWK is that there are many implementations and they are not always compatible with one another:

https://www.gnu.org/software/gawk/manual/html_node/Other-Ver...

I have ported AWK scripts from legacy Unix systems to Linux and ran into incompatibilities that required some adjustments to the scripts.

Curious: what systems have AWK, but do not have Perl?

In practice, on actual systems, you're likely to encounter nawk (the One True Awk), gawk (GNU Awk), mawk ("Mike's AWK", a fast awk), and Busybox's AWK.

There are other variants, yes, but in virtually every case these are fully POSIX compliant and/or have a POSIX mode.

(And in truth, gawk is the only non-fully-POSIX awk I've encountered --- it extends standard AWK with asort and the "'" formatting modifier (which prints localised htousands separators in numeric data).

Programmes written for any one awk, if using POSIX features only, will run on any awk.

Many small / embedded systems (think routers, stock Android, or any POSIX-only Unix variant) must have awk, but often don't include Perl.

You'll also find variants of Perl, though the relative stasis of that language make this less an issue now than in the '90s and aughts.

Perl came out in the late 80s and by the mid 2000's was really on its way out? I hired for my last perl position in around 2006.

Just saying, your definition of "kids today" could well include a decent portion of developers under 45 years old. Referring to this cohort repeatedly as "kids" is also a little cringe.

Did you really just reply to a comment that used the phrase "Kids Today (tm)" and try to interpret it as a genuine insult? The inability of this community to understand straightforward humor amazes me. Dude, it was a joke. And yes, I was calling mid-career professionals "kids". Deliberately. Because I'm old. And it's funny.

I'm one of those "kids these days" but did actually learn to program Perl at some point, and I generally prefer AWK. Perl is a large and complex language, I don't need it that often, and I'm not smart enough to keep remembering all of it.

Now, if I would get hired as a full-time Perl developer and spent 2 years developing Perl: it would perhaps be different. But that's not the case, and isn't for most people.

For better or worse, Perl sees a lot less usage than it once did; I rarely encounter it "in the wild" and don't even have it on my laptop because nothing needs it.

Does any OS besides Windows not ship with Perl? Even on Windows, I'd assume anyone programming has WSL set up, which means you have Perl.

Not being snarky, why not python over perl? what makes perl better for scripts?

what makes perl better for scripts?

Having an implicit line- and field-splitting loop for standard input with a couple of command-line switches. (Awk doesn't even need switches, but is cumbersome if you need initial state.) This covers a lot of use-cases. Also, very compact and powerful regular expressions.

Perl is much more terse for one-liners and has much more built-in for doing text processing in scripts. Stuff like implicit read loop, field separation, etc. I would say they're suitable for different jobs: if a perl script grows beyond a hundred lines (you can do a surprising amount in that space!), then Python may be the right tool.

Perl is also much more of a known target: some version of it exists on basically every single Unix, and the language really hasn't changed that much in the past decade. I have SSH'ed into multiple CentOS 6/SLES 11 (released 2009, and granted mostly to rescue data off them) servers in the past 2 years, and perl is just much more of a known target to write things against than whatever python release is on that system.

Perl is a progression of that particular environment. It is a superset of shell/grep/awk/sed.

A shell command works exactly as you would expect copied literally inside a backquote. With all the other goodies of a real programming langauge.

Doing this in Python (to me atleast) seems unnatural.

By the time you figure out which env you need to be using with python, you’ll forget why you needed it.

If all you're doing is text processing that Perl can do out of the box, you probably only need the Python stdlib.

awk is more intuitive for sure, for a regular javascript coder

Remove perl, have less security issues. Some scanning tools flag it, too. Awk is found in more places, in my experience.

Sorry, but that's ridiculous. Any general purpose programming language is a vector for bugs and security problems, but come on: you're genuinely trying to say that a kludge of bash+sed+awk is objectively more "secure" than a single perl script to solve the same problem?

In the case of awk, actually yes, it is safer. The reason is that awk is a very limited language. It has only enough functionality to provide text matching and substitution. It is very difficult to use awk to do anything of high security risk, compared to a language like perl.

But awk is never used alone. You don't solve whole problems with awk, you squish it into a script with a bunch of other junk. My point is that you're making an apples-to-oranges comparison. Sure, "awk" isn't the problem, but "bash" is, and bash is undeniably a more error-prone language than perl. You surely agree with that much, right?

And if you disallow "bash" for security reasons, where does that leave "awk" in the category of useful tools? See my point?

You’re right. But the alternative might be bash + perl or just bash. Or none of them. Perl is anyway the first one to go.

Is it completely gone, or rather just for you, blocked by sysadmins who know Perl is the magic pixie dust for total control, and want to keep it for themselves?

In Windows-land, compare how PowerShell access may be restricted, and you won't be allowed to run macros in Office, all while your computer is "managed" by a horrible hodge-podge of PowerShell and VBA scripts that make Perl code look like high literature.

Just use awk for what it was designed: text search and substitution. You can run shell scripts along with awk, but that is clearly not what you should be doing if you want to design secure systems. The first rule of security is not to abuse your tools.

> It has only enough functionality to provide text matching and substitution.

Gawk at least can do a lot more than that. Reading and writing files, network communications, and run arbitrary shell commands, for example. It's certainly not as powerful as perl but it's also not limited to just text matching and substitution.

Edit: figured I would provide some examples. Here's an http server and a first person shooter in gawk. Maybe not so practical but they show some of gawk's capabilities.

https://github.com/kevin-albert/awkserver

https://github.com/TheMozg/awk-raycaster

There is a virus written in awk that infects other awk scripts[0]. And according to wikipedia, the language is Turing complete.

[0]: https://github.com/SPTHvx/ezines/tree/main/dc5/CODES/Perfori...

Reply on news.ycombinator.com