Hacker News Reader

Comment by Game_Ender | Hacker News Reader

For single files or monitoring other pipe based operations (like network transfers) pv [1] is also a useful tool.

1 - http://www.ivarch.com/programs/pv.shtml

Using pv:

  gcp() {
    cat "$1" | pv | cat > "$2"
  }

useless use of cat

    pv < "$1" > "$2"

pv reads content from a file by default, so that can actually be reduced further to:

    pv "$1" > "$2"

Which will actually give better results this way, since it'll read the size of the file and use that to provide an estimate even.

Even if the cat at the beginning of a pipe isn't strictly necessary, it still helps readability to have everything flowing from left to right.

I also advocate this "useless" use of cat, but you can write the redirection before the command:

    <"$1" pv >"$2"

The bigger problem is that using cat destroys the file size information.

I find pv useful when moving MySQL databases between servers.

mysqldump -u user dbname | pv | mysql -u user -h some_remote server dbname

to copy a tree of files:

  tar c SOURCE_DIR | pv | tar xv -C DEST_DIR

  # on source computer
  tar c SOURCE_DIR | pv > /dev/tcp/192.168.1.2/4567

  # on destination computer computer
  nc -l -p 4567 | pv | tar xv -C DEST_DIR

For anyone curious as I was why the /dev/tcp/... redirection works, the Bash manual contains the following:

    Bash  handles  several filenames specially when they are used in
    redirections, as described in the following table:

      /dev/fd/fd
         If fd is a valid integer, file  descriptor  fd  is
         duplicated.
      /dev/stdin
         File descriptor 0 is duplicated.
      /dev/stdout
         File descriptor 1 is duplicated.
      /dev/stderr
         File descriptor 2 is duplicated.
      /dev/tcp/host/port
         If  host  is a valid hostname or Internet address,
         and port is an  integer  port  number  or  service
         name,  bash  attempts  to open a TCP connection to
         the corresponding socket.
      /dev/udp/host/port
         If host is a valid hostname or  Internet  address,
         and  port  is  an  integer  port number or service
         name, bash attempts to open a  UDP  connection  to
         the corresponding socket.

pv is a great tool and one of the first things I install on new machines. I hate having no idea how long an operation is going to take. For example, loading a SQL dump

pv foo.sql | mysql

Reply on news.ycombinator.com