The SQL layer in CockroachDB
github.com
github.com
PS: Should any cockroach contributors have opinions on limitations & evolution of the protocol: I'd be interested.
Overall I think it is a very good protocol. I can't come up with any gripes I have with it, nor do I remember any "it would be nice if" moments when adding support for some of the various parts of it. There's certainly some difficulty in some of the more complicated parts, like managing all of the type hinting back and forth for prepared statements. But in general I found the documentation to be complete and well done.
The one thing that we did have trouble with was figuring out the exact formats for all types when using the binary encoding format. We wrote some code to sniff real Postgres servers so we could figure out what was going on. It would have been easier, obviously, if it was documented, but this wasn't a huge deal since it is likely a rarely needed thing.
Thanks for replying!
> Overall I think it is a very good protocol. I can't come up with any gripes I have with it, nor do I remember any "it would be nice if" moments when adding support for some of the various parts of it. There's certainly some difficulty in some of the more complicated parts, like managing all of the type hinting back and forth for prepared statements. But in general I found the documentation to be complete and well done.
Cool. I've more complaints than you in that case :) Although drivers, especially libpq (no pipelining, no mixed binary/text results), are more frequently an issue.
One thing I was wondering whether you might also be interested is something like 'commit identifiers' (WAL LSNs in pg's case) for slightly relaxed consistency models. That allows things like committing on one node, and when later doing queries on replicas specify that data needs to have replicated at least up to that identifier.
> The one thing that we did have trouble with was figuring out the exact formats for all types when using the binary encoding format. We wrote some code to sniff real Postgres servers so we could figure out what was going on. It would have been easier, obviously, if it was documented, but this wasn't a huge deal since it is likely a rarely needed thing.
Yea, I think that's a fair complaint. I wasn't around yet when the current binary format was devised (2003ish IIRC?), I don't know how we ended up deciding that code was the documentation for that.
The Go lib/pq driver is some pretty old Go code and has numerous similar problems. We try to keep it moving along but it's got some stuff that makes it hard to work on.
> One thing I was wondering whether you might also be interested is something like 'commit identifiers' (WAL LSNs in pg's case) for slightly relaxed consistency models. That allows things like committing on one node, and when later doing queries on replicas specify that data needs to have replicated at least up to that identifier.
CockroachDB is always serializable. It is linerizable on individual keys. We've discussed having a causality token before that could be passed to other transactions or connections that I think would allow for more linerizable guarantees. If I'm understanding your question correctly then yes, I think these would be very useful to us. (I've asked someone more familiar with our consistency model to address this more because I'm not confident in the accuracy of my reply.)
Also on the subject of the protocol, we've run into a need for a (server-initiated) ping message (https://github.com/cockroachdb/cockroach/pull/10188).
There was some discussion around this before, and the idea was basically to just include some text-payload after COMMIT. PG would produce something like "COMMIT 343/3f0037a0", but the spec for would just be text. Some drivers might do something with the knowledge that it looks like an LSN, but that's hard to avoid.
> Also on the subject of the protocol, we've run into a need for a (server-initiated) ping message (https://github.com/cockroachdb/cockroach/pull/10188).
Maybe I'm missing something here (I admittedly only skimmed ticket and things it references), but shouldn't server triggered tcp keepalives be sufficient for this case? In postgres that's configured using tcp_keepalives_idle/tcp_keepalives_interval/tcp_keepalives_count. Several drivers (including at least libpq and pgjdbc) can do the same to protect against vanishing servers.
Doing keepalives on the pg wire protocol level has some disadvantages, but also some issues. Namely you've to be ready to send ping/pongs forth/back in almost any protocol state. Not necessarily easy if there's partial writes and such.
So I do think "pgwrite" level pings make some sense, it's just not exactly straightforward to add them :/
My expectation would be that at some point you'd want to be able to optionally relax that (on a per sql transaction or table basis maybe?). E.g. on a "counter" table you instead might want do use something CRDT like, because you don't want to incur the latency of having to contact other servers. On an append only table without problematic constraints, it can be very interesting to do so too, particularly useful for logging & timeseries data. Not saying that has to have any sort of priority ;)
I was looking at using/extracting their pgwire implementation to provide an SQL-like interface to the in-memory state of a particular application we use, this document will make it way way easier.
It would be great to see this document evolve into a guideline rather than a "this is how most of the code works today" document.
As someone downthread said, it's not a consumer product and doesn't need to be marketed like one.
If you don't think the name is hurting adoption you live in fairyland. A place where startups get extra karma for being edgy and mentally challenged.
cockroach is a word rarely encountered my non native English speakers. Expect them to only know the "cock" part, not sure what "roach" could mean.
Thus this database is often interpreted as CockDB :D
It this deeply tied in to the rest of cockroachdb's architecture, or does it talk through a simple generic KV store interface?
[1]: https://github.com/cockroachdb/cockroach/tree/master/pkg/kv
[2]: https://github.com/cockroachdb/cockroach/tree/master/pkg/sql
[3]: https://paperhub.s3.amazonaws.com/dace52a42c07f7f8348b08dc2b...
You guys are ruining my lunch time.
proprietary auto sharting algorithm makes me laugh every time.
but i guess no one got the joke , because you know apple is like a famous tech company and mongodb is a popular nosql db
so apple+mongodb = mangodb .. got it .. no .. :(
"Roaches on Open Water! CockroachDB on DigitalOcean" https://www.cockroachlabs.com/blog/roaches-on-open-water-coc...
Next up - Annual cockroachdb conference: roach motel.
They can keep thinking the name is not a problem, or they can decide to grow up and change the name.
Only naive engineers who know nothing about branding/marketing will think this is OK. It's fine if they're just doing their own thing, but it's a whole different story if they're operating a business
The name is one thing that anyone can have an opinion about, the average person can't really speak to the internals of the database, but anyone can look at the name and say "I don't like it and wouldn't want to say I have 'cockroachdb' in my stack".
The very fact that it comes up every time there's a story about the database should be a pretty strong sign that the name is overshadowing the product. If it was named after an animal that doesn't have nearly universal disdain like "Koala-db" or "dragonfly-db", or even "ant-db", then I doubt people would be bringing up the name every time it's mentioned
But the reason that people don't complain about the name every time git is brought up is because git comes up in conversation a lot. CockroachDB is also a really bad name - I do agree - but the reason almost every thread about it on here is partially derailed by chatter about its name is, I think, simply because it doesn't come up all that often.
That's stretching it a bit too far. Just because it's a "mild oath" in British english, doesn't mean it's bad name for rest of the world. I for one have never heard about it and don't think git is a bad name at all.
Cockroach, on the other hand, I don't know of anyone who would get a good image in their mind hearing it.
So you think if people keep saying cockroach a lot, suddenly cockroach will magically become some nice clean creature in people's mind? That's like saying if you keep saying "ShitDB", someday people will think it's a good name.
I'm saying these people are doing their own technology disservice by making the brand difficult to talk about and take seriously.
It's their choice, but there's a difference between "stupid names don't matter, look at Google!" and "stupid names don't matter, look at FuckDB! RapeDB! CockroachDB!" (I know i'm going extreme with this but just trying to get the point across, that "stupid" names and "offensive" names are completely different things)
If you still disagree with me, I don't really have much intention of trying to persuade you nor care much about this company succeeding. I'm just saying it's a bad business decision to make it hard for your customers to advocate for your product. If there was a company with the same technology and they had a normal name, they would probably get much better traction (assuming that their tech is unique and useful).
Something like that. People will associate the name with the database, not the insect. This does happen, over time you almost forget the the original meaning.
Some examples: when somebody says Oracle or Delphi, I'm thinking databases and Pascal successor, not ancient Greek stuff. When somebody says Pascal, I'm thinking programming language not the French mathematician. Fedora is a Linux distribution (and also a type of hat), Twitter is a social network (and also a type of bird vocalization) and so on.
Cockroach not only has negative connotation, but the negativity scale is pretty strong.
Bring me a single person who will say they get good emotion when they hear "cockcroach". Sometimes you need to be stubborn about your business decisions, but if you're being stubborn about naming your business after a word most of the population will get negative feelings the first time they hear, then you're doing it wrong. It's not even a subjective thing, with a bad name, people are less likely to talk about your brand than if you had a positive name.
Or, for a not-exactly-elegant name: Ubuntu 4.10 Warty Warthog. You are launching a brand-new Linux distribution, why not call it after a warty wild pig. Worked OK for them.
"Cockroach" on the other hand, like i said, I know of absolutely NO one who would get positive emotion upon hearing that word. Just like "rape", or "nazi" don't evoke positive emotion for anyone.
First impressions actually matter in human interactions, and when you're named after one of the most disgusting pests known to man, that's definitely not helping your case.
Sorry, not everyone (especially management and others you have to explain it to who aren't rational super-men) get a good impression on a technology you're peddling... it's associated literally with cockroaches. Good luck selling that to them. What's next, "Cancer-DB"?
Because "aid" means to give assistance and that database is here to help you ;)
"Mongo" is a norwegian/swedish(?) slang for a retarted person. It's derived from the word "mongoloid" which was an person with Downs or a similar syndrome. The latter word is not commonly used anymore.
If you think you are not affected. Would you try a product named "octopussy"? Would you deploy it across your organization and gives presentation about it?
P.S. Octopussy is a real tool that solves a hard problem I forgot about.