Hacker News Reader

Comment by Aissen | Hacker News Reader

Just look at anything in the "moreutils" package. Just great stuff. Excerpt:

- ifdata: do not parse the output of ifconfig/ip anymore. Just use this tool.

- sponge: when you need to overwrite an input file at the end of pipe. sponge will wait for the pipe to end before overwriting, preventing any data loss

- vidir: edit a given dir with your EDITOR. Awesome for mass renames/deletes.

- ts: add timestamps to a command

- parallel: C implementation of GNU parallel (in perl). Very small, very fast, just does the core (running commands in parallel), and does not try to take over xargs.

There are others of course.

View on HN

Although these tools are great, I'm afraid these are of limited use, as these aren't preinstalled but must be explicitly installed.

For personal use these are great. But if you write a shell script to be executed on different machines where you can't install anything, these tools won't be available to you.

And if you don't have that requirement, i.e. you can install everything, just install appropriate Perl/Python/Ruby libraries and use a proper scripting language.

(BTW, I tend to assume that Python is installed on almost all modern Unix systems by default, hence I prefer writing portable tools in Python instead of portable Shell code.)

Is that really a problem anymore? Not being able to install things was common in multi-user thin-client type systems, but these days with containerisation and VMs, if you're allowed to execute a script, you're probably allowed to install new packages

No?

Sadly, no.

It is often possible to install things locally, i.e. not using apt. The idea that everyone has sudo privs is unrealistic. A version of apt that added a per user subset would be very useful.

This is actually one of the things that made me work on rootless containers (https://rootlesscontaine.rs). It is a pity that everyone assumes you have sudo access to all of the machines you want to run code on.

There are other ways. fakeroot/fakechroot for a poor man's "container" (I cringe inside when I say that). Cross compilation + copying binaries. Or compiling on target with an appropriate ./configure --prefix.

Nix and Guix allow unprivileged package operations.

I regularly run "one-off" stuff in a `guix environment` that includes all the required packages for that command, without polluting my user (or system) profile.

For example, I recently wanted to transfer a file over HTTP from one machine to another, and did not have the "python" executable in PATH. So I started it in a `guix environment` and "containerized" it (using user namespaces) just for show:

    guix environment --container --network --ad-hoc python -- python3 -m http.server

That starts the HTTP server from the current directory, but the process can not see anything else from the "real" system.

Why assume python? It's huge compared to bash or perl. Lots of minimal configuration operational systems won't have python. More likely than go/ruby/node though.

I had the impression that more an more system/initialization scripts are written in Python, at places where shell and perl were used in the past.

And having a system using Python on startup is a pretty good indicator that Python will be preinstalled.

However, I see that the situation may be different in specialized distros.

If you're on RH derived systems, you're going to have a python 2.6 installed for rpm and yum to work.

This bit us in our ass with centos 6.8 , when another package required python 2.7... If you install 2.7 directly, your system fails with a thousand cuts. A chroot is needed for that, unfortunately.

chroot is unnecessary. You just install python 2.7 in its own directory and run it from there. The system-provided python 2.6 and your desired version of python should co-exist.

As a general rule, never try to replace any program or library supplied by your distribution.

I had this problem recently. The solution was to install Python 2.7 as a software collection: https://www.softwarecollections.org/en/scls/rhscl/python27/

This installs it to /opt/rh, and you can run commands that need them with "scl enable python27 mything" or by sourcing /opt/rh/python27/enable in a script to set up PATH, etc.

You don't need to install go. It has the runtime thing built into the compiled executable.

Same with C. However, the binaries are then architecture dependent, so not really portable either.

Nevertheless, statically linked binaries are indeed a good alternative, and provide good-enough portability in a wide range of cases.

You do if you want to modify the code... Which is kind of the point.

Is Python that much bigger than Perl?

And from your list, Go doesn't really belong there, it outputs statically linked binaries. You can just plonk them anywhere.

Most of the time, I try not to care. It's packaged everywhere, so either install it or ask the script users to do so.

When you're using Python/Perl/Ruby you'll often have to use pip/cpan/rvm for dependencies, and you're back to square one.

One of my favorite moreutils tools is combine[1]. It allows you to compare the contents of text files with boolean operations. For example, want to know what lines are in file1 but not file2? Use:

    combine file1 not file2

1. http://manpages.ubuntu.com/manpages/zesty/en/man1/combine.1....

You can use comm from coreutils for that too: `comm -1 file1 file2`

Yep, but you'll have to read the man everytime to figure out if you need -1, -2 or -3, or a combination of those. combine gives you a better interface with boolean operators you know.

The 1st column is the lines unique to the 1st file, the 2nd column is the lines unique to the 2nd file, and the 3rd one is the lines that are not unique. -1, -2, -3 allow to disable those columns. Pretty easy to remember.

Ah, HN downvoting neutral comments again and I don't like it, but I have a policy of not upvoting anything out of pity.

Ahaha, then I guess I don't use it that often. I'll try to remember this mnemonic.

PS: do not worry about karma ^^

The comm manpage[1] says that the files have to be sorted. In combine, they do not.

1. http://manpages.ubuntu.com/manpages/zesty/en/man1/comm.1.htm...

Reply on news.ycombinator.com