Hacker News Reader

Comment by sillysaurus | Hacker News Reader

Would anyone mind doing me a favor by explaining xargs in more detail? I've tried learning it a couple times but I always seem to forget the primary situations in which it's useful. Thank you in advance!

View on HN

Xargs takes a newline separated list and maps the list to a command.

    find ./ -name '*.log' | xargs rm

Finds all log files and map them to 'rm' commands. e.g. if it finds system.log and rails.log it will run the command `rm system.log rails.log`.

xargs will automatically do things like break up very long lists into multiple command calls so that it doesn't exceed the maximum number of arguments a command can have.

Other useful things about xargs are '-P <NUM>' which lets you run the same command in parallel. I use this with curl to do ghetto benchmarks.

The next major flag is `-n <NUM>` which changes the number of arguments per command call. e.g. `-n 1` will run the command per argument passed to xargs.

And the last flag I commonly use is `-I {}`. This sets `{}` as a variable which contains the arguments. (This also forces `-n 1`). This is useful for things like:

    find ./ -name '*.log' | xargs -I{} sh -c "if [ -f {}.gz ]; then rm {}; fi"

    find ./ -name '*.log' | xargs rm

Only do that if you know exactly what '*.log' will expand to (i.e. don't use it in scripts and avoid using it on the command line). This is because the delimiter for xargs is a newline character, but filenames can have a newline character in them. This can lead to unexpected results.

Almost everywhere I see xargs used, find ...-exec {} ; will work as well and find ...-exec {} + may work even better.

    find ./ -name '*.log' -print0 | xargs -0 rm

Fixes that issue and xargs is far more efficient, it doesn't launch a new process for each line like exec does, but far more importantly, xargs is generalizable to all commands so you only have to learn it once; exec is just an ugly hack on find, you can't generalize it across all commands; xargs is much more unixy.

> it doesn't launch a new process for each line like exec does

Exec doesn't either if you use "+" as a terminator:

    find . $(options) -exec command {} \;

executes one command per match,

    find . $(options) -exec command {} +

executes a single command with all matches

> exec is just an ugly hack on find

Obvious and complete nonsense, -exec is both an important predicate of find and a useful and efficient tool.

-print0/xargs -0, now that is a hack.

You apparently missed the part where I said "but far more importantly".

No, that's just your opinion and you're entitled to it so I don't care, the rest is factually incorrect.

Oh, I agree on the + thing, I wasn't aware of that option, however, it doesn't make exec any more generalizable across commands which is what matters.

Wanting to remove the files was so common that some finds have it built-in. Avoids even the overhead of -exec rm {} +.

    find -name '*.log' -delete

True, but the issue isn't removing files, the issue is generalizing a command for mapping output of a command to another command. rm was just a simple example. xargs is far more useful than simply deleting files.

Removing files is a common need coupled with find yet many readers don't know of -delete; I was pointing it out. That doesn't weaken xargs's valid uses. You seem a little defensive? :-)

Too much reddit perhaps. I use the delete switch myself regularly.

Note that xargs actually takes a whitespace-delimited list. This often leads to problems when a filename has a space in it. To fix it, you should either use:

  find ... -print0 | xargs -0 ...

or:

  ... | xargs -d'\n' ...

It's most common use is to take some stuff on stdin and then use those as arguments to a command. Here is a fancy way to do `ls * `[1] using xargs:

    find . -print | xargs ls

`find` dumps the results to stdout, the pipe shuttles `find`'s stdout to `xargs` stdin, `xargs` uses its stdin as arguments for `ls`.

(Don't run this in your home directory)

[1] Pedants will realize this is actually equivalent to `ls .* *` as hidden files are found by `find`.

Maybe you'll call me a pedant, but you should be aware that `find .` is not equivalent to `ls .* *`. The find command starts at the indicated directories (. in this case) and lists each file and directory within it, recursing into subdirectories. You can use things like -type, -[i]name, and -mtime to filter the results, as well as -mindepth and -maxdepth to constrain the traversal.

Note also that "-print" is the default command for find, so you can leave it off. Other commands include -print0 (NUL-delimited instead of newline-delimited) and -exec.

Reply on news.ycombinator.com