555 karma · joined July 25, 2011
Perhaps UPS/Fedex could act as a verification service. For a small fee, the UPS store employees could photograph the contents of the package before it gets mailed, check if the electronics turn on, write down the serial numbers, etc. and send a report to eBay.
[1] https://en.wikipedia.org/wiki/Occupy_movement_hand_signals [2] https://www.youtube.com/watch?v=HaqpAlvDKsI&feature=youtu.be... [3] https://www.signingsavvy.com/sign/APPLAUSE/7950/1
You've probably heard that the difference between the GPL and the AGPL is whether they apply to access over a network or not; but you seem to have misinterpreted how that works. The GPL says that if you modify GPL-licensed software and distribute compiled binaries to users, you must also distribute the source code under the GPL. The AGPL says the same thing as the GPL, except that making the modified software available over the network is treated the same as distributing the modified software as a compiled binary.
> At NASA, RethinkDB is radically simplifying how we provide real-time services in support of Extra-Vehicular Activity. -- Collin Estes, Director of Software Engineering, Chief Architect - MRI Technologies Inc. - NASA / ESOC
My guess is that RethinkDB failed financially because they had a lot of users but not a lot of customers. Most users weren't buying support contracts.
The client drivers are licensed separately under the Apache license. Only the database server itself is AGPL. Read the last question on the rethinkdb.com FAQ: https://rethinkdb.com/faq/
Imagine if 99.99% of queries take 1 millisecond, and the remaining 0.01% of queries take 6000 seconds. Then the mean is 0.6010 seconds and the standard deviation is 59.997 seconds.
(Muphry's law strikes again...)
Wasn't this fixed in RethinkDB 2.1? Or are you referring to something different?
[0] http://www.bloomberg.com/view/contributors/ARbTQlRLRjE/matth... [1] http://www.bloomberg.com/view/articles/2016-05-06/hedge-fund...
However, a Raft cluster will get stuck if half or more of the members are permanently lost. RethinkDB offers a manual recovery mechanism called "emergency repair" in this case. When the administrator executes an emergency repair, RethinkDB discards the old Raft cluster and starts a completely new Raft cluster, with an empty log and so on. However, some servers might not find out about the emergency repair operation immediately. So we would end up with some servers that were in the new Raft cluster and some that were still in the old Raft cluster. We want the ones in the old Raft cluster to discard their old Raft metadata and join the new Raft cluster.
The process of having those servers join the new Raft cluster is managed using the epoch_t struct. An epoch_t is a unique identifier for a Raft cluster. It contains a wall-clock timestamp and a random UUID. When the user runs an emergency repair, the wall-clock time is initialized to max(current_time(), prev_epoch.timestamp+1). When two servers detect that they are in different Raft clusters, the one that has the lower epoch timestamp discards its data and joins the other one's Raft cluster. The UUID is used for tiebreaking in the unlikely event that the timestamps are the same.
So the clock isn't being used as a trusted source of truth; it's being used as part of a quick-and-dirty emergency repair mechanism for the case where the Raft cluster is permanently broken. The emergency repair mechanism isn't guaranteed to preserve any consistency invariants (as the documentation clearly warns).
In the Raft protocol, the leader sends AppendEntries; the follower writes the log entries to persistent storage, but doesn't apply them to the state machine yet; the leader sends another AppendEntries with a higher commit index; and then the follower applies the changes to the state machine. In RethinkDB's case, the "state machine" is the B-tree we store on disk. One of the guarantees we provide is that if the server acknowledges a write to the client, then all future reads should see that write; and we perform reads by querying the B-tree. So we can't acknowledge writes until they've been actually written to the B-tree, and we can't start writing to the B-tree until after the write has been committed via Raft. So that's where the latency would come from if we were using Raft to manage individual documents. We considered a couple of ways to work around this. One would be to make reads check both the B-tree and the Raft log; but that makes the read logic much more complicated. Another would be to start writing to the B-tree as soon as we put the write in the Raft log. The problem is that Raft expects to be able to roll back parts of its log at any time, and our storage engine's MVCC capabilities aren't good enough for that. The hybrid approach allows us to have good latency without major rewrites of the existing storage engine.
The other reason is that RethinkDB allows tables to be split into shards, and the number of shards can be changed while the databases is running and accepting queries. We considered having one Raft instance per shard, but we would have needed to modify the Raft algorithm to allow splitting and merging Raft instances while they are running and accepting queries. (This is approximately what CockroachDB is doing [1].) But the Raft algorithm is really tricky to get right even when you're not trying to modify it, and we wanted to stick as closely as possible to the official algorithm to minimize bugs. The approach we ended up going with allows us to have the performance and convenience of live resharding without having to modify the Raft algorithm.
In response to your question about consistency guarantees: RethinkDB gives users several options for trading off consistency and latency. The default is to acknowledge writes only once they're safely on disk on a majority of replicas, but to perform reads only on the primary replica (leader). This doesn't give perfect consistency; if the leader fails over, reads that hit the database around the time of the failover might see outdated data, or they might read writes that were rolled back as part of the failover. Unfortunately, the only way to get stronger consistency guarantees is to wait for a majority of replicas to acknowledge the read, which makes performance much worse. We offer a safe-but-slow mode for reads, but it's not the default because the performance is so bad. We also offer fast-but-unsafe modes for writes, for users that want better latency and are OK with losing the last few writes in the event of a failover. See the documentation [2] for more information.
[1] https://github.com/martinrusev/rethinkvsmongo-benchmark/blob...
As of the time I posted this comment, the blog post still seems to be comparing indexed MongoDB operations against non-indexed RethinkDB operations. Under those conditions I'd expect RethinkDB to be at least 1000x slower than MongoDB. The fact that he's finding that RethinkDB is only 3x slower than MongoDB makes me think that there are still other major problems with this benchmark.
> No, but at least he tried...
It's true that the author tried; but that doesn't change the fact that people are going to read this blog post and assume that the numbers are at least approximately correct. As a RethinkDB employee, it really frustrates me to see RethinkDB being judged according to benchmarks that are conducted so carelessly that they are essentially random.
I think this is the fourth time in the past year that I've seen a third party try to benchmark RethinkDB and get something wrong. Maybe we need to start a "best practice" of checking in with the maintainers of a project before publishing benchmark results about the project.
The solution is to re-architect RethinkDB so that it can reconfigure a table even if there's a disconnected server. This is a pretty big project, but we're working on it, and it will probably ship around April or May. We'll also include server-side auto failover at that time, because it's easy once this problem is solved.
We don't have any plans to support direct access from web clients. The main problem is security. RethinkDB doesn't have any notion of users or access privileges, so any client can e.g. drop a table. We might add this eventually, but it's low on the priority list. See <https://github.com/rethinkdb/rethinkdb/issues/266> for more discussion.