Hacker News Reader

Comment by joshbaptiste | Hacker News Reader

Yep, I see too many senior people violate UUOC/UUOG ie..

  cat file | grep bar | awk '{ print $1 }' | sort | uniq

which can be

  awk '/bar/ { print $1 | "sort -u"}' file

The best resource if you really want to be a better shell citizen (assuming you use Bash) overall is #bash on irc.freenode.net and Greg's bash wiki.

http://mywiki.wooledge.org/BashFAQ

Honorable mention also for #awk on irc.freenode.net also which has taught me a lot.

View on HN

I would still use the former.

Chances are, I'm not going to remember awk's somewhat weird, more unfamiliar syntax -- everyone knows about cat, everyone knows about piping, those two go very well together.

Using cat with pipes typically linearizes $x operation in our minds, it makes it easier to come up with solutions, and makes it easier to change them as modification is needed.

Using cat with pipes typically linearizes $x operation in our minds, it makes it easier to come up with solutions, and makes it easier to change them as modification is needed.

Yes, definitely. Also it's easier to build up from simple to more complicated pipelines that way, testing whether each iteration does what you expect it to.

I find those kind of examples a little unfair as what you're actually doing is comparing the syntax of different shells/languages (the Bourne Shell vs awk). You could write examples in Perl or Python as well - to prove how pointless awk and sort is as well if you really wanted. In fact you could even rewrite sort in awk.

While I disagree with most of the comments on here about how:

    cat $file | grep "pattern"

is easier to read than

    grep "pattern" $file

(in my opinion, if you're a sysadmin then you should know how grep works and thus the optimised solution shouldn't be any harder to read. But at the end of the day, whatever gets the job done).

However I do draw the line at your example because you're now comparing apples to oranges and suggesting that everyone should learn an additional programming language on top of memorising every standard Unix/GNU/BSD command and their flags.

What about this?

    <$file grep "patter"

It keeps the flow of information left-to-right, but also avoids UUOC.

It's silly when set up like that. But I still prefer the separate invocations -- they become building blocks. It means I can replace the "cat file" with "echo testdata", it reads nice, and for more complicated scripts, it's easy to format across lines:

    zcat /var/log/somdeamon/* \
     | grep "simple event like maybe 'error'"
     | grep "complicated date expression or whatever"
     | some other filter
     | summation?
     | maybe sort and uniq

I can start that with a simple (z)cat of one file (rather than the thousands of lines I might want to work on) -- and it doesn't require much arcane knowledge of each of the tools used, for each step.

I do agree that awk might be a little under-appreciated, but if you really feel that way, maybe you should just use snobol.

To me a "wasted cat" clearly states intention: get/feed data. Then the rest of the steps either filter, sum, or manipulate data.

The final thing that makes the "waste"-argument silly, is that Linux[1] has pretty lightweight processes. I'm not sure what the state of the various BSDs are, but it'd surprise if at least FreeBSD hasn't gotten a lot better on this too. Either way, one would think that any text pipeline would be io-bound, not limited by the number of forks...

[1] http://bulk.fefe.de/scalability/

Your first line of code is much better. It optimizes for clarity, presenting the programmer's intention in a simple logical sequence. It also likely optimizes for programmer time and mental expenditure.

Are you sure about that, what if file contains the equal sign?

  $ touch foo=bar
  $ file foo=bar
  foo=bar: empty
  $ awk '/bar/ { print $1 | "sort -u"}' foo=bar
  
  
  
  
  
  
  bar
  
  screw this!
  ^D
  bar
  $ awk '/bar/ { print $1 | "sort -u"}' <foo=bar
  $

I don't think it's fair to get awk involved. It's a scripting language, not a single-purpose utility.

Why not:

  awk '/bar/ { print $1 }' file | sort -u

Reply on news.ycombinator.com