scripts, shell

Some xargs

Rafacas has already mentioned it, but xargs is sometimes much more useful than what it looks like. Two examples come to mind:

  • Way too many files for rm or ls. It may well happen that a script has generated more than 10000 files in the same directory (it was your friend, not you, I know). If you try and rm * in there, you will be in trouble. Ditto if you simply want to count them with ls | wc -l. However, the following works:
    $ find . -type f -print0 | xargs -0 ls | wc -l
    

    (or rm instead of ls). It works because there is no globbing taking place. The -print0 option forces find to end lines using a zero \000 code, so that spaces are not taken into account as ‘field separators’. The -0 option of xargs is used exactly in this context: it sets 0 as the field separator to expect. Actually, if your friend produced normal filenames (without spaces or tabs inside), both options are unnecessary. Like in the next example.

  • You want to use either the contents of a file or the result of a (probably long) pipeline to do a simple task. Say files is a list of files to be just created. Then
    $ cat files | xargs touch
    

    does the job.

There is a way to include trailing arguments (stdin is usually appended at the end of the code after xargs): use the -J command-line option:

$ ls -1d [A-Z]* | xargs -J % cp -rp % /temp/capitals

The above line will copy all the files and directories (at the present directory) starting with a capital letter to /temp/capitals. The -J option is followed by a string (in the example ‘%’) whose first (and only first) appearance will be replaced by the corresponding item from the pipe. In the example, if the files and directories in question are just AFILE.txt and A_dir, then the line above is equivalent to the two lines which follow:

$ cp -rp AFILE.txt /temp/capitals
$ cp -rp A_dir /temp/capitals

As usual, man xargs is your friend. By the way, I’d thank anyone who is able to explain the -I option to me.

1 Comment

  • On 04.06.11 chaz said:

    The difference between -I and -J can be illustrated with an example.

    Let’s say I have a list of files that I want to touch:

    $ cat list_of_files
    file name one
    file name two
    file name three
    file name four
    file name five

    Using -J doesn’t handle this case correctly:

    $ cat list_of_files | xargs -t -J {} touch {}
    touch file name one file name two file name three file name four file name five

    $ ls -1
    file
    five
    four
    list_of_files
    name
    one
    three
    two

    But using -I does:

    $ cat list_of_files | xargs -t -I {} touch {}
    touch file name one
    touch file name two
    touch file name three
    touch file name four
    touch file name five

    $ ls -1
    file name five
    file name four
    file name one
    file name three
    file name two
    list_of_files

    I’m still not clear on the reason why -I and -J are both available, as -I is more robust, at least as far as I can tell. And you could replicate the behavior of -J with -I, if desired, using -n1:

    $ cat list_of_files | xargs -t -n1 -I {} touch {}
    touch file
    touch name
    touch one
    touch file
    touch name
    touch two
    touch file
    touch name
    touch three
    touch file
    touch name
    touch four
    touch file
    touch name
    touch five

    And furthermore, you can use the replacement string fed to -I in double-quotes, whereas it doesn’t work exactly right using the replacement string fed to -J:

    $ cat list_of_files | xargs -I {} echo “Hello from {}”
    Hello from file name one
    Hello from file name two
    Hello from file name three
    Hello from file name four
    Hello from file name five

    $ cat list_of_files | xargs -J {} echo “Hello from {}”
    Hello from {} file name one file name two file name three file name four file name five

speak up

Add your comment below, or trackback from your own site.

Subscribe to these comments.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*Required Fields