Hacker News Reader

Comment by thaumaturgy | Hacker News Reader

I'm mostly with you (except for your last sentence). I've picked up a lot of languages over the years, and can fall back into most of them pretty fast, but sed and awk especially are hard. On the occasions that they really are the best tool for the job, I get actual books out and set aside a window of a few hours to plod through it.

I think the problem is that they fall into an uncanny valley between "simple tool" and "verbose programming language"; they are really terse languages wearing a simple tool's sheepskin.

View on HN

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it.

And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the awk/sed can't hold its own as a general purpose programming language, now I am writing bash code, a personal sin.

Awk/sed made sense for the time and place they are from, but things have changed.

Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now."

Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?

I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason.

Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk. I got into sed a while ago but never use it. Awk, together with sort and uniq sometimes, is all I need.

And it is interactive. You have an input. Print that out and look what needs to be done. Write the first transforming step in Awk. See what that did. Add the next step. See what that did. Extremely interactive. After a few minutes I have a one-liner (often 2 or 3 wrapped lines) that does exactly what I want, and easy to adapt to other similar things. I have very large bash history enabled so they stay Ctrl-R-accessible.

Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new.

(I might be wrong about "everything" but that's what bomb-tossing is all about)

Welcome to the discussion. I like it, I haven't perled in a while but does that mean you can evolve from a one liner to a full program using all of perl using this technique?

I suppose it's a "pathway to perl" in the sense of a mild gateway drug. Perl is definitely one-liner-friendly. Every time I get into Perl I have to relearn it all over again (which is apparently not an isolated phenomenon), so I don't have great suggestions for digging further. I just habitually reach for perl -pe about 3/4 of the time.

"Be kind. Don't be snarky."

"Please don't sneer, including at the rest of the community."

https://news.ycombinator.com/newsguidelines.html

I really wish all these bash scripts would be replaced with python scripts.

Yeah I redid some scripts on my team for my own usage because our team bash guru maintained them and didn't want anyone to change them. So I did my own versions in python. Sure they were mostly 2x+ as long but they were very easy to read and modify and test. The bash guru finds out and brings it up at a team meeting trying to shame me, but it ended up with my boss asking the team, and they voted on using my scripts in lieu of the older bash scripts as I had written a few unit tests around them and the rest of the team understood them without digging out unix tomes.

I have done something similar, what I really found worked well was to wrap the bash scripts in python to instrument them and if they just operate on data, you can even spawn another command and run them both at the time confirming that the output matches.

Great way to have confidence in a port and provide for a fallback path. Eventually the new code will take over the old code and the old code will wither and die.

It seems like when a bash script gets over say 50 lines or so I start doubting whether something should be a bash script :) . I saw an IRC client written entirely in bash once and it was quite a resource of esoteric bash scripting haha.

> Awk/sed made sense for the time and place they are from, but things have changed.

Don't do that. Do not assume something is of no use because you don't know how to use it. Double that recommendation for tools that have widespread use. There is a reason they're used everywhere.

In the case of awk/sed/perl: Nothing beats them for composition in one-liner text processing pipelines.

plus, they exist in the vast majority of unix like systems. You don't have to install anything. They are just there and available to use. Even using them in a simplistic manner is light-years ahead of trying to do stuff manually.

I have moved on to structured data, awk and sed handle line oriented text formats.

Don't assume I don't know how to use it. :)

One line verses ten, I'll take the more readable thing over the APL.

>Performance is comparable

Can't find links now, but fairly certain were stories in past about how switching to AWK from other tools increased performance

I gave up on them except for the very most common uses of them for slicing/dicing text, after the most essential basics I just turn to python and suck it up. Learning to pipe things in/out in python is a great skill to have in these circumstances. I think perl is the king of this but I'm not going back to that... never again :)

Reply on news.ycombinator.com