CockroachDB Skitters into Beta
cockroachlabs.com
cockroachlabs.com
Besides, they could've called it PoopDB and I still would use it if it survived a nuclear blast and had joins. I couldn't care less about the name. But apparently I'm in a minority...?
It's like that test where men were shown pictures of women and asked to (a) pick the one they thought was the most beautiful and (b) guess which one others would pick most often. Result: they pick the girl-next-door for (a) but think others will all go for the pornstar-look.
Yeah, I'm not sure what's up with all the hand-wringing. Dunno if it's a generational thing, or just a peculiarity of this particular echo chamber. [0] None of folks at the jobs that I've worked cared about the name of a tool as long as it was good.
[0] I do take comfort in the fact that the hand-wringing is a tiny slice of all of the conversation attached to the submission.
That would be Keith Richards and cockroaches (sorry had to). So two names are up for grabs for the competition.
Oh, and the client drivers page should probably also list pgx (https://github.com/jackc/pgx), which is a nice, fast PostgreSQL client that doesn't use the not-as-nice database/sql package.
___
† The name being the only exception; I don't have bug phobia at all, but cockroaches are just icky.
I noticed this: CockroachDB is not suitable for use cases involving joins (yet)
Well guys, that's the kicker. If you've got joins, you've got SQL. The selection language isn't so important if you can't do set relations.
I'm sure you're going to rock this. Can't wait to see how you do it!
Right now, I'm most interested in seeing RethinkDB see more traction... it seems to have most of the features I look for, and I, for one don't mind more programmatic queries, and the management interface is really second to none at this point... MS-SQL, I'd probably consider a close second.
I'm going to have to disagree. The German HyPer DBMS has a lot more interesting ideas. Cockroach is just distributed MVCC from the 1980s on top of a key/value store.
MemSQL also has a new LLVM architecture written by the HipHop VM inventor that they poached from Facebook.
Yeah. Not so much. I can't think of anything that used the 2 phase transactions over independent consensus groups (aka spanner) pattern from the 80's. Likewise there was nothing like Yabandeh transactions in 1980 because no one actually understood serializable snapshot isolation then (but several projects mistakenly thought they had, which was Yabandeh et all's entire motivation).
HyPer is neat work, but that it is interesting does not mean CockroachDB is not also.
And according to HyPer home page, HyPer doesn't look distributed, which is another big difference.
On the other hand, the author (or authors) seem completely unresponsive about some things. Their Homebrew package's Ruby support is still broken a year after reporting it [1]. The Homebrew package is also ancient (1.4.4 instead of 1.8.1); I did a PR [2], which requires that the authors to roll a release of their fixes to HEAD of libpo6 [3]. All of this has been ignored for some time.
I'm just worried that the project, which seems to be maintained by just one developer, is languishing. Very little commit activity, lots of open issues, lack of attention, etc.
[1] https://github.com/HyperDex/homebrew-hyperdex/issues/10
Also... why would you want to live WITHOUT ANY JOINS nowadays?! Unless the scale requirements actually prohibit it, and let's face it, 99% of what people are doing is not even close to "big data" on modern hardware, and no, your 1TB DB is not big data and you can handle it with Postgres or a powerful graph db just fine on a pretty cheap VDS if you stay way from AWS/Azure etc. that get nowhere near the performance/price of renting "dedicated metal" on mid-term deals. I'm getting closer tho thinking along the lines of: not even plain JOINs are enough, I want at least Postgress-like recursive joins if not full-fledged infinite-depth graph traversals! I know the budget/hardware allows is, so give it to me, stop the whole "scalability" bullshit!
* Document or SQL Storage * Integrated Full Text Search * Can transparently survive a node going down (load balancer or client driver support) * Highly consistent by default, or allow operation level quorum settings * DR options with minimal lost writes * Transaction support * Hosted * Affordable for small businesses with low average load, but including the occassional peaky or seasonal issues (large bulk imports, auto/consumer/product/etc-show traffic spikes for a few days, etc)
Cloudant meets everything but Transactions. But that's a big one.
Most databases you might think of don't satisfy the Hosted option. And considering how easy it is to loose other items on the list through misconfiguration or maintenance issues, that's a lot more of a core feature than might be obvious for a small shop. It's great if node failure is transparent in theory, but if it doesn't work in practice it may as well not exist.
Most of the rest don't integrate full-text-search. That's huge. It means a ton of concerns move from the database vendor, to you, the application developer. Any database that doesn't (at least plan to) integrate full-text-search is one with a much higher development and operations cost.
And the number of databases that promise to (eventually) deliver distributed SQL? There was 1 a few years ago (AFAIK) that met most of those: FoundationDB. Never to be heard from again. :-)
I don't need a GraphDB for my CMS-like sites. That's just not a very good fit. And working without JOINs (though it sounds like they're on the roadmap here) is not as onerous as you'd imagine for many sites, and it often brings along the side-effect of forcing you to write a much more efficient/fast product against denormalized data.
So any database that promises to check most of those boxes is worth keeping an eye on (IMO). Though the things that kill most solutions candidacy for me are:
- lack of integrated Search
- lack of Hosted/Managed option
We call it a data layer, not a database, because it's a higher-level document data model layered on top of Postgres (with pluggable data stores that can be added at runtime; we hope to support other backends such as Redis and Cassandra and maybe CockroachDB if it's a good fit) and Elasticsearch (also intended to be pluggable), similar to how TitanDB is implemented.
It supports transactions — atomic multi-document updates within a single data store that supports this — fine-grained CRDT-style document patching, update streaming, versioned schemas and a fine-grained permissions system. The query language isn't SQL, but our own relationship-oriented, vaguely GraphQL-like query language that allows querying on anything that Elasticsearch can express, as well as complex joins and aggregations. Since Elasticsearch is eventually consistent and all queries go through it, we aim to provide some consistency-tolerance semantics when a client does need strict consistency in combination with queries.
We're not quite ready to publish the code, but drop me line (email in profile) if you want to be notified when it becomes available.
I dislike CouchDB too for many of the same reasons, but when selling it to clients, I first point out that I know the db is not winning marketing awards and then go into why it's awesome and the proper solution.
I'll have to do this for CockroachDB too, but it's quite a bit worse than CouchDB IMO.
Calling a product a "mouse" must have seemed like a strange decision at the time. Now we don't think twice about it. My point is that people get accustomed to things over time.
While I don't care about the cockroach name, I think RoachDB would be a suitable "change" if they have to.
Maybe _you_ don't think mice are gross, but maybe you're not a representative sample.
I never got the impression that mice are seen as gross.
Mice also chew holes in walls, cupboards, pipes, and wiring, sometimes to disastrous effect (e.g. house fire [2]).
You can pretend to be all macho and unconcerned by mice, but the fact is, they're awful disease-carrying rodents, and you ought to avoid them and get rid of them if they're in your house.
[1] http://pestcontrol.about.com/od/diyrodentcontrol/a/Rats-And-...
Of course, if the majority of people would see an actual mouse on their cupboard they would not be so happy - me included.
Of course, literally the first time I ever saw a cockroach was when I visited New York a few years ago. I was riding the escalator down to the subway and the cockroach was riding the up escalator next to me! Off to its next great adventure, I'm sure.
1. Backing up your data is somehow an Enterprise feature?
2. No pricing anywhere on the site, which usually means:
2.1. Sales calls. Nothing I hate more in this world than sales calls.
2.2. $$$$$$$ (which is fine, but I like to know roughly how much $$$$$ before 2.1.)
As far as functional indices, I don't see that here:
http://docs.memsql.com/docs/create-index
What I mean is specifying a WHERE clause on the index.
Seriously, though, this looks like a very interesting project. Until now, I had not been aware it supports SQL. I think I have to give CockroachDB a try in my next toy project.
It sounds like they try do build something very ambitious. To my knowledge none of the commercial database vendors (except maybe Tandem back in the day?) have succeeded in building a distributed/replicated SQL-based database engine with strong consistency and high availability. I know MSSQL supports hot-standby/failover, but AFAIK that does not cover all the scenarios CockroachDB seems to want to address.
https://www.cockroachlabs.com/docs/frequently-asked-question...
https://www.cockroachlabs.com/docs/secure-a-cluster.html
(Much thanks to Go's extensive standard library: https://github.com/cockroachdb/cockroach/blob/master/securit... )
I've lost count on the number of projects that leaves "securing the server" as an exercise to the reader (about as useful as those error prompts that helpfully ask you to "contact the server administrator" (I AM THE ADMINISTRATOR!)).
I understand why not using TLS doesn't give an error, although I'd prefer it if it was more work to set up an insecure instance (eg: --force-no-tls --force-no-auth) -- but what cockroach does here is pretty good, and AFAIK best-in-class from those which it is natural to compare to (in all fairness, projects like postgresql a) doesn't do too bad, and b) have a lot of legacy cruft -- the old (current) assumption is of course that if you want TLS, you'll use your secure in-house CA for everything. Which even when running Microsoft AD is in my experience way too complex for most to bother with).
Another (bad) example here is openssh, that has had decent support for certificates rather than keys for a long while now, and yet I've yet to see anyone that appear to use ssh certs in anger (Myself included, it's high up on the infinite todo-list).
One other great example of making the best of what awful tools are available for securing communications is the Caddy web server, that comes with built-in support for letsencrypt: https://caddyserver.com/
That said, while I think the docs are pretty good here, a note that leaving an unecrypted, ca-key lying around is a bad idea, might be worth a mention even in the quick-setup. At least as it stands, it's reasonably easy for someone with a working knowledge of TLS/certificates to guess which parts should be secret (all the keys) and which parts should be kept air-gapped (the ca-key).
Maybe a link to how cockroach parses/verifies certs would be nice too, for those that do have a working internal CA -- along with a little info on how/if certs integrate with authenticating nodes (eg: will my printer.example.com x509-cert allow my hacked printer to join the cluster, if it is signed by the same CA?).
Despite all the comments, I'm really liking what I see so far -- I'll be keeping an eye on this project!
TLS is ok if you need to secure something that speaks HTTP (or IMAP etc) -- but I wonder if ssh certs might not be the lesser evil for something like what cockroachdb needs. Perhaps esepcially now as self-signed/"home grown" x509 certs are being relegated to the background by modern browsers -- so you'd end up needing a "real" cert for the status-server to present securely to a web-client, which might not match the wish for every node to have its own keypair.
That's what they expect from their db... the ability to survive huge failures whilst maintaining data integrity.
Super excited to hear about CockroachDB's progress - and their decision to add SQL support I think was smart.
However, I can't go without critiquing their marketing of "Scalable" and "Strongly Consistent", especially when they reference horizontal scaling. Strongly Consistent systems bottleneck around a centralized master, or around complex consensus algorithms like PAXOS or Raft. CockroachDB has previously written this off with "we're using Google's Spanner algorithm" but this makes them vulnerable to a Split-Brain failure (which contradicts their other goal of "Survivability"), for anybody interested in this line of research I recommend googling Kyle Kingsbury "Aphyr" "Call Me Maybe" Jepsen Tests. With that said, you can't just toss horizontally "Scalable" and "Strongly Consistent" into the same blog because they are fundamentally at odds with each other - the physics just don't work. To achieve this you only have two options... either (A) vertically scale the Master, or (B) make incredibly technical sharding decisions (where no two pieces of data that depend upon each other can be in separate shards). The first (A) is not the definition of being scalable, because a single machine can only scale so much (limits of the physical system). And (B) cannot be generalized, which would require every company building on top of CockroachDB to either have the expert knowledge of predicting the exact sharding schema in advance (maybe why they are moving to SQL?), or have to depend upon expensive consulting options (perhaps provided by CockroachDB, after a startup has already built itself around using the free open source product?). The third option is that they are over-generalizing their database trade-offs, and that you cannot actually get horizontal "Scalability" and "Strong Consistency".
That said, I know the CockroachDB guys are smart, so I'd love to hear the more nuanced/detailed explanation. I'm just trying to keep your guys' marketing and promises in check with the reality.
The rest of your comment ignores a couple decades or so of very extensive research into scalable distributed transactions.
Design documentation is here if you care to read it: https://github.com/cockroachdb/cockroach/blob/master/docs/de...
I have read the design document, and know how Spanner works. This isn't the first time I've asked about this, I had the 2nd highest upvoted comment on the initial CockroachDB announcement ( https://news.ycombinator.com/item?id=9660339 ) which means it is many many more people than just me that are concerned about these claims.
Unless you can show me exactly what research contradicts my critique, your response has no merit and doesn't address my concerns. You can't handwave magic appeals to authorities without citing them. So my questions still stand.