Comment by Ultimatt | Hacker News Reader

Comment by Ultimatt | Hacker News Reader

Skip awk, use perl....

The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE will then Evaluate an expression from the command line looping over the input line-by-line.

    alias glorp='perl -aF"/\s+/" -nE'

So now we have the command `glorp` to play with which has more familiar syntax than awk and all of CPAN available to play with!

    $ [data is generated] | glorp '/Something/ and say $F[2]'

We have access to any Perl module by putting -MModule::Name=function after the command, the following will parse a JSON record per line and glorp out what we wanted:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'say decode_json($_)->{hello};' -MJSON=decode_json
    world
    cat

Maybe you are used to using curl too. There is a nice web framework in Perl called Mojolicious (http://mojolicious.org) that provides a convenience module called 'ojo' for command line use. So grabbing the summary sentence from Wikipedia articles is as straight forward as below. Notice Mojolicious lets us use CSS selectors!

    $ echo -e 'grep\nawk\nperl' \
      | glorp 'say g("wikipedia.org/wiki/$F[0]")->dom->at("#mw-content-text > div > p")->all_text' -Mojo

Here's the equivalent for Ruby:

    alias glorp='ruby -ane '

    $ [data is generated] | glorp ' ~ /Something/ and puts $F[2]'

Or:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'puts JSON.load($_)["hello"] ' -rjson

(Of course Ruby got the -a autosplit-mode and the -n assumed 'while gets(); ... end' loop from Perl along with $_ and $F, so it's very intentional that they're similar)

Awesome thanks for sharing this! I was too lazy to give a Ruby example alongside.

Somewhat related nodejs self plug: Use nip https://github.com/kolodny/nip

    $  echo -e 'this\nis\na\nwhatever foo' | nip 'return /whatever/.test(line) && cols[1]' # foo

And Ruby regexes are amazing.

I thought that they were amazing because they were just like Perl's. Are there any differences?

Sorry, but I strongly disagree.

I can think of two very strong reasons to prefer awk over perl:

* busybox implements awk, so its (usually) available even on embedded platforms

* awk syntax is way clearer than perl, anyone that has a few experience in C + shell can figure out what is being done with some googling

awk is also powerful enough to separate output to different files, accumulate inputs, etc. And if you need anything more complex, there are tons of other languages to choose from then (including perl).

what's the reason of using -F option? -a defaults " " separator, which already emulates awk behavior. If I'm not wrong, only difference between perl -aF"/\s+/" and perl -a is treating of leading whitespace in lines

To show you could use other separators defined by any regex you like.

Hey, it would be awesome to have something like this for Python!

Not really possible in Python for anything overly useful/complex because of the significant white space, and regular expressions are not a first class object in the language. ipython is the equivalent solution I suppose.

However, Ruby completely inherited this aspect of Perl and is another good candidate. It's just even old systems have a perl installed that will perform the awk like functionality.

There is "Pyed Piper" aka `pyp`

https://code.google.com/archive/p/pyp/

"Pyp is a linux command line text manipulation tool similar to awk or sed, but which uses standard python string and list methods as well as custom functions evolved to generate fast results in an intense production environment."

That's cool, but I feel like you're unlikely to run into it very often in the wild? The vast majority of Linux systems come with ruby/perl out of the box. Plus if Python did support this sort of hackery it would quickly garner most of the bad press Perl has over the years :'(

Who cares? It's on pypi (https://pypi.python.org/pypi/pyp/2.11) so just `pip install pyp` and then you have it. You're probably not going to want to distribute shell scripts that rely on pyp, but if you're distributing scripts you can just distribute scripts written entirely in Python and use pyp for your own one-liners.

The lack of something like Perl/Ruby's -n and -p switches is a barrier as well. Those are handy because they automatically work for either piped input, or files named on the command line.

Of course, you can do all of this in Python, just not as a one liner.

I wrote a little tool called "pyline" about a hundred years ago... I still reach for it often when I'm in a hurry, and don't have time to futz around with awk/sed.

https://code.activestate.com/recipes/437932-pyline-a-grep-li...

That's pretty cool, thanks!

Check out osh: https://github.com/geophile/osh

so is the intention to be python equivalent of awk

There was an awk/perl thread yesterday too.

Reply on news.ycombinator.com