Hacker News Reader

Most of the time it's much better (as in, faster) to just use cat (or pv, to get a nice progress bar) for writing a file to a block device, because it streams, and lets underlying heuristics worry about block sizes and whatnot.

So:

  cat foobar.img > /dev/sdi

will stream the file rather than what dd does, i.e. read block, write block, read block, write block and so on.

Usually I also lower the vm.dirty_bytes and vm.dirty_background_bytes to 16 resp. 48 MB (in bytes) beforehand, which limits the buffer sizes to those amounts. Else it will seem that the progress bar indicates 300MB/s is written, and when it completes you still have to wait a really long time for things to have been written out. Afterwards I restore back vm.dirty_ratio and vm.dirty_background_ratio to respectively 10 and 5 - the defaults on my system.

I wish that all of those projects, tutorials etc. that explain how to write their image to a block device, like an sdcard, would start advise using cat, because there is no reason to use dd, it's just something that people stick with because others do it too.

I only use dd for specific blocks, like writing back a backup of the mbr, or as a rudimentary hex editor.

View on HN

-> % python test.py 2>&1 | python test.py COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME [snip] python 24150 someone 0u CHR 136,0 0t0 3 /dev/pts/0 python 24150 someone 1w FIFO 0,8 0t0 72772851 pipe python 24150 someone 2w FIFO 0,8 0t0 72772851 pipe python 24150 someone 3w CHR 5,0 0t0 1049 /dev/tty COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME [snip] python 24151 someone 0r FIFO 0,8 0t0 72772851 pipe python 24151 someone 1u CHR 136,0 0t0 3 /dev/pts/0 python 24151 someone 2u CHR 136,0 0t0 3 /dev/pts/0 python 24151 someone 3w CHR 5,0 0t0 1049 /dev/tty someone@arch-two [01:29:00] [~] -> % echo $ZSH_VERSION 5.2 someone@arch-two [01:29:16] [~] -> % cat test.py import os import sys import subprocess with open("/dev/tty", "w") as f: subprocess.check_call(["lsof", "-p", str(os.getpid())], stdout=f)

strace dd if=10M of=junk read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512 strace cat 10M > junk2 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072 write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 131072