Hacker News Reader

Comment by LiveTheDream | Hacker News Reader

I use this command very frequently to check how often an event occurs in a log file over time (specifically in 10-minute buckets), assuming the file is formatting like "INFO - [2014-08-27 16:16:29,578] Something something something"

    cat /path/to/logfile | grep PATTERN | sed 's/.*\(2014-..-..\) \(..\):\(.\).*/\1 \2:\3x/' | uniq -c

results in:

    273 2014-08-27 14:5x
    222 2014-08-27 15:0x
    201 2014-08-27 15:1x
    171 2014-08-27 15:2x
    349 2014-08-27 15:3x
    230 2014-08-27 15:4x
    236 2014-08-27 15:5x
    339 2014-08-27 16:0x
    330 2014-08-27 16:1x

This can subsequently be visualized with a tool like gnuplot or Excel.

View on HN

sed -E for awk or grep -E regexes.

Useless use of cat?

"Don't pipe a cat" is how I'm used to describing what you're talking about -- it may have been a performance issue in days past, but these days I think it's simply a matter of style. Not that style is not important.

This was drilled into me back in the usenet days. If you see a cat command with a single argument it's almost always replaceable by a shell redirection, or in this case just by passing the filename as an argument to grep. If you're processing lots of data like in the article there's no point in passing it through a separate command and pipe first.

I think people like reading cat thefile | grepsedawk -opts 'prog' from left to right, and that they think the only alternative is grepsedawk -opts 'prog' thefile.

But there's grep <thefile -opts 're'. I like that one best; it reads the same way you'd tend to think it.

But when you're interactively creating your pipeline, having the cat at the very beginning can save you some shuffling around. For example, while your second command is be grep and you're passing your file directly to grep, you might then realize you need another command before grep. So you'll have to move that argument to the other command. With a useless cat in front, you just insert the new command between cat and grep. It doesn't really cause harm.

Reply on news.ycombinator.com