105 karma · joined October 27, 2010
Neither log shipping (copying WAL files one by one) nor streaming replication (sending a stream of WAL) works by sending queries. WAL segments are 16MB by default, and the default archive_timeout is 0, not 1 minute (and the archive timeout is not applicable to streaming replication anyway). There is also nothing "adaptive" about the replication—when there is no traffic, there will be ~no changes, and when there are changes, they will be sent to the replica.
I don't understand what the comment is suggesting used to happen in periods of no activity that made replication unusable, but it is also probably incorrect, and has nothing to do with the write amplification problem.
What it sends is not the queries, but a logical description of the changes to each row that were made by running the query. So an UPDATE that changes N rows would generate N changes to be applied to the corresponding rows (usually identified by primary key) on the logical replica, not a single update that had to be "re-executed".
You can see it under /proc/$pid/fd. For example, if you run "sleep 600|wc -c" (just to give yourself some time to poke around), you can see:
$ ls -l /proc/2760791/fd
total 0
lrwx------ 1 ams ams 64 Mar 28 13:35 0 -> /dev/pts/10
l-wx------ 1 ams ams 64 Mar 28 13:35 1 -> pipe:[54074122]
lrwx------ 1 ams ams 64 Mar 28 13:35 2 -> /dev/pts/10
$ ls -l /proc/2760792/fd
total 0
lr-x------ 1 ams ams 64 Mar 28 13:35 0 -> pipe:[54074122]
lrwx------ 1 ams ams 64 Mar 28 13:35 1 -> /dev/pts/10
lrwx------ 1 ams ams 64 Mar 28 13:35 2 -> /dev/pts/10
54074122 is the (virtual, i.e., not disk-based) inode of the pipe. You can get substantially the same information from lsof:
$ lsof -E -u ams|egrep 'sleep|wc'|grep pipe
sleep 2760791 ams 1w FIFO 0,13 0t0 54074122 pipe 2760792,wc,0r
wc 2760792 ams 0r FIFO 0,13 0t0 54074122 pipe 2760791,sleep,1w
static char *pipefs_dname(struct dentry *dentry, char *buffer, int buflen)
{
return dynamic_dname(dentry, buffer, buflen, "pipe:[%lu]",
d_inode(dentry)->i_ino);
}
That said, it is not based on socketpair any more. sys/kern/sys_pipe.c says:
/*
* This file contains a high-performance replacement for the socket-based
* pipes scheme originally used in FreeBSD/4.4Lite. It does not support
* all features of sockets, but does do everything that pipes normally
* do.
*/No, they do not (nor on the modern BSD kernels, as far as I can tell). The Linux pipe(7) manpage says (under "Portability notes"):
«On some systems (but not Linux), pipes are bidirectional:
data can be transmitted in both directions between the
pipe ends. POSIX.1 requires only unidirectional pipes.
Portable applications should avoid reliance on
bidirectional pipe semantics.»
I also looked at the pipe implementation in Minix, which is a (non-trivial) variant of John S. Dyson's implementation that the BSDs share. It is implemented as a server (in the microkernel sense), so there's quite some added complexity there in handling "vmount"s and locking, but there are still some familiar elements of the code too, such as the "put it all together with flags" code in create_pipe().
https://github.com/Stichting-MINIX-Research-Foundation/minix...
For something even further along these lines, there's also the pipe implementation from Plan9, which at first glance felt so unfamiliar that I wasn't sure I was looking in the right place:
Thanks, you are perfectly right. What I found charming was that the 3E pipe.2 manual page contains «word^H^H^H^H____» written out by hand in the roff _source_. The 4E one switched to using ".it word" instead.
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V3/man/man2/pi...
Another charming little quirk: the 6E kernel's falloc() does «printf("no files\n")» if it fails to find a free file structure.
Glad you enjoyed it. :-)
> I have considered doing something totally ridiculous like setting up a Raspberry Pi with a camera and machine vision software to watch the LED displays on these devices to glean status information. Silly, but...
Ha! I actually have a PoE camera pointed at the display of my UPS. Here's what it looks like right now: https://toroid.org/misc/ups-display.jpg
Notice that horizontal blank row of dead pixels halfway down the right side of the display? The one that makes "54.6" look like "51.6"? That gap defeated my naïve five-minute attempt to use image recognition to extract the battery voltage.
But I should also clarify that my long-running fsck isn't always the result of an unclean shutdown. There's something about my combination of iSCSI+crypttab+NFS that causes fsck to be run too often—even if I shut down the machine cleanly while the NAS is running, it usually decides to fsck when it comes back up.
Something to investigate next winter, perhaps.
No, it's very far wrong. (I'm a Malayali.)
I started by reading the NaCl source and all the introductory material available—the web site, the two NaCl papers ("Cryptography in NaCl", "The security impact of a new cryptography library"), and quickly reviewed a couple of other papers (e.g. to understand deterministic encryption and D-H key exchange). I think the biggest problem I had was to understand nonce generation and handling properly. The "Cryptography …" paper does contain some advice that I ultimately implemented. But I had to think very hard about what it said before I was confident that I was doing what it said. For example, it says:
«…the nonce can be chosen as a simple counter: 0 for Alice’s first packet, 1 for Bob’s first packet, 2 for Alice’s second packet, 3 for Bob’s second packet, 4 for Alice’s third packet, 5 for Bob’s third packet, etc. Choosing the nonce as a counter followed by (e.g.) 32 random bits helps protect some protocols against denial-of-service attacks. In many applications it is better to increase the counter to, e.g., the number of nanoseconds that have passed since a standard epoch in the local clock, so that the current value of the counter does not leak the traffic rate.»
I managed to figure it out, but I would certainly have welcomed a more detailed explanation, and would have been very happy to have help from the code to do the right thing.
In contrast, the cryptography functions were easy enough to figure out and use (with the C API). The only mistake I kept making was specifying the secret key first and the public key second in all my function calls. Once I got used to doing it the other way around, it was fine. Zero-padding the messages was slightly ugly, but I didn't develop any especially strong feelings about it. (Aside: I actually ended up using TweetNaCl, but of course all the documentation is the same.)
I'm very pleased with the resulting code, anyway.
P.S. I looked at libsodium, but greatly preferred the unadorned library.
As a former dog owner, I found the dog example in this article especially evocative, because I've seen what an effort it can be for dogs to sit still when they're told.
(Minor aside: the extraordinary overuse of emphasis made this article much harder to read.)
Did you read the judgement yet? There's no point discussing anything without knowing the facts of the case.
«Seems quite obvious that if find yourself in that photo spot, and a red bus passes by, you immediately think to make it pop out with such a post processing trick.»
If that's all it had been, the judgement makes it clear that the result would not have been infringing.
The finding hinges on the fact that the infringing photograph was not an independent work, but was created based on knowledge of the claimant's original photograph, without drawing inspiration from (or knowing about) other similar works, a series of which are analysed. In no way does it mean that any photograph of a red bus in London would infringe on the claimant's copyright; in fact, this is specifically denied, and various ways in which the defendant could have created a non-infringing photograph are discussed.
I find it's often instructive to read actual judgements rather than what is reported about them in the press.