I'm glad it runs fast, and I hope that all OS utilities are optimised (and tested, of course!) instead of making their source code pretty. The fact is, most people want to use programs, not read them.
That really doesn't make any sense, /dev/zero should be at least as fast as yes.
I agree, all I remember is that when I tried it, /dev/zero sometimes sucked performance-wise. I can't recall the exact circumstances as it was some time ago, and could have been on any of Linux/FreeBSD/SunOS/HP-UX/IRIX - perhaps it was the fastest common way at the time?
On a recent x64 Linux, /dev/zero seems plenty fast enough now:
$ dd bs=8k count=819200 if=/dev/zero of=/dev/null
819200+0 records in
819200+0 records out
6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.331137 s, 20.3 GB/s
$ yes | dd bs=8k count=819200 of=/dev/null
819200+0 records in
819200+0 records out
6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.959551 s, 7.0 GB/s$ pv < /dev/zero > /dev/null [ 16GiB/s]
But the version of yes using vmsplice() is even faster than that on my machine.
pv < /usr/bin/yes > /dev/null
doesn't seem to work properly. FWIW I get 330MiB/s vs 8.4GiB/s for /dev/zero.[Incidentally first I've heard of pv but I've known about dd for a decade or two].
yes | pv > /dev/nullcat /dev/zero | pv > /dev/null
uses a pipe like the 'yes' line above, but runs substantially slower on my computer than the redirect-only version.
yes | pv > /dev/nullFrom pv's man page:
> Its standard input will be passed through to its standard output and progress will be shown on standard error.
> Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(
The Windows/DOS command line has the same concepts[0], though it's probably used less often: by default a process has 3 FDs for STDIN (0), STDOUT (1) and STDERR (2).
At a shell
* you can feed a file to STDIN via <$FILE (so `</dev/zero pv` will feed the data of /dev/zero to pv), the 0 is optional
* you can pipe an output to a command (other | command) or the output of a command to an other (command | other)
* you can redirect the STDOUT or STDERR to files via 1> and 2> (the "1" is optional so 1> and > are the same thing) (you can redirect both independently)
* you can "merge" either output stream to the other by redirecting them to &FD so 1>&2 will redirect STDOUT (1) to STDERR (2) and 2>&1 will redirect STDERR (2) to STDOUT (1), you can combine that with a regular redirection so e.g. `command >foo 2>&1 ` or with a pipe (`command 2>&1 | other`)
And you can actually create more FDs to use in your pipelines[1] though I don't remember ever seeing
[0] https://support.microsoft.com/en-us/help/110930/redirecting-...
I want to use safe programs, and programs with readable code are more likely to be properly audited.
I actually checked just now, and it looks like you'd be making twice as many system calls with /dev/zero compared to generating the data locally:
strace dd count=1 bs=512 if=/dev/zero of=test
open("/dev/zero", O_RDONLY) = 3
dup2(3, 0) = 0
close(3) = 0
lseek(0, 0, SEEK_CUR) = 0
open("test2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
dup2(3, 1) = 1
close(3) = 0
read(0,"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
close(0) = 0
close(1)
While 'strace yes > test2' is just a constant stream of write() calls.The difference matters if you're benchmarking e.g. some new SSD compared to a tmpfs on a machine with 100+ GB of RAM. It's always better if the tools have less overhead, because the comparison is more meaningful.
Also consider that it can be faster to write to a local network than to disk. I've never done it, but I imagine that the kernel's not going to want to deal with your /dev/zero calls if it's spending all of its time writing to a 10GB switch. I can imagine some very specialized storage servers that could spend most of their time writing from memory buffers to a network switch, or if you're troubleshooting a slowdown in the networking itself.
$ strace yes > /tmp/test2
[various initialization steps]
write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
[...]
To measure everything, I started up a whole new shell, expecting to see a 'write(1, ...)', a 'read(1)', and a 'write(f, ...)': $ strace -f sh -c 'yes > /tmp/test2'
[various initialization steps]
[pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
[pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
[pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
[pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
[...]
How does this possibly work? File descriptor 1 is supposed to be the terminal, not a file! Of course, the magic of file redirection: open("/tmp/test2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 # Open the file, get FD 3
fcntl(1, F_DUPFD, 10) = 10 # Copy FD 1 (the terminal, STDOUT) to FD 10 temporarily
close(1) = 0 # Close original FD 1
fcntl(10, F_SETFD, FD_CLOEXEC) = 0 # Close FD 10 (the terminal, copy of STDOUT) when exec() is called
dup2(3, 1) = 1 # Copy FD 3 (the file) to FD 1 (STDOUT)
close(3) = 0 # Close FD 3 (the file's original descrptor) cat /dev/zero|pv >/dev/null
gives me roughly 2/3 the speed of yes. (5 GB/s vs 7 GB/s) Plus yes gives you an arbitrary string, instead of only zeroes. pv > /dev/null < /dev/zero </dev/zero pv >/dev/nullpv </dev/zero >/dev/null
which is a common way of doing it (for any command with any inputs and outputs, not just the above ones), i.e.:
command < input_source > output_dest
All three pv command invocation variants, the one here and the two above, work. And it becomes more clear why they are the same, when you know that the redirections are done by the shell + kernel, and the command (such as pv) does not even know about it. So in all three cases, it does not see the redirections, because they are already done by the time the command starts running (with its stdin and stdout redirected to / from those respective sources). And that is precisely why the command works the same whether it is reading from the keyboard or a file or pipe, and whether it is writing to the screen or a file or pipe.
but here's todays Useless Use of Cat Award http://porkmail.org/era/unix/award.html
https://news.ycombinator.com/item?id=14414610
But in that case one could probably say that Gnu awk is just not very good at input handling (as Mawk doesn't appear to benefit from an extra "cat").
[edit] That's actually inaccurate (and badly expressed), see comments below.
When I was writing raspbian images to SD cards to use on a raspabery Cat and DD took within a few seconds of each other on an operation longer then a minute. Since then I have been using cat where I could, but I didn't think to right down the numbers though.
https://news.ycombinator.com/item?id=14414610
Note that cat+gnu awk was faster than just gnu awk - but mawk was faster still (reading a not entirely small file).
And in a similar vein of gp comparing Gnu and Openbsd, note that openbsd cat is a little more convoluted than the simplest possible implementation (at least to my eyes):
https://github.com/openbsd/src/blob/master/bin/cat/cat.c
https://github.com/coreutils/coreutils/blob/master/src/cat.c
(That is, Gnu "cat" and OpenBSD "cat" are less different than Gnu "yes" and OpenBSD "yes").
yes(){ while :; do echo "${1:-y}"; done; }
Python: import sys
while True:
if len(sys.argv) > 1:
print(sys.argv[1])
else:
print("y")
And if you don't need the features of `yes` and only need a loop that prints 'y', then there really is hard to beat the simplicity of: while:; do echo "y"; donehttps://github.com/openbsd/src/blob/master/usr.bin/yes/yes.c
It's essentially line for line identical to your python code...
For readability purpose it is easier to go through each lines of the python program than the OpenBSD C code. Its not massively different, but its distinguishable enough that I would choose the python version if I wanted to maximize readability, minimize syntax requirement and did not want to use shell script.
The Shell function is in my view the superior choice if the audience is a programmer than know the shell script syntax. It is just a single loop and is written in the environment that the program is intended to be used in. The only drawback there is the speed.
... and python. Don't forget to count python.
Just because you say so?
Not a huge bit of work to be sure but for a non-commercial project you might have trouble finding a volunteer who cares about that tedium.