Why Go Was the Right Choice for CockroachDB | Hacker News Reader

Why Go Was the Right Choice for CockroachDB

cockroachlabs.com

62 pointsscapbi34 comments

The article seems to make a lot of very dubious claims about Java. Especially when they started this project, Java performance was much better than go (Those double-digit second GC pauses weren't a concern with go?), and the Java garbage collection remains much more sophisticated than even current bleeding edge go.

Also the claim that you have to add extra interfaces to Java is completely false. There's no reason at all you need to add pointless interfaces in Java. Java's type system is strictly more powerful than go's type system. This was actually a design choice in go to make the language easier.

Java generics are also a huge boon when writing something like a database.

Modern C++ also would have been an excellent choice for writing a database, and would have been much more productive than go.

> The article seems to make a lot of very dubious claims about Java.

I was also stumped by this confidence in that Go is supposed to be better than Java performance-wise and I do think that the claims warrant some proof.

However:

> Modern C++ ... would have been much more productive than go.

I'm not up to date on current C++, but this seems like a claim much like the ones you objected to. Could you elaborate on how C++ is a more productive language (in this case?)?

"What remains to be seen, of course, is how all of this Go code will perform."

They're pretty clear and honest that Go's performance characteristics are an unknown and not the reason it was chosen.

There is a clear bias.

When talking about Java:

> JVM garbage collection is notoriously expensive.

> Java’s known performance issues made it unappealing.

> However, in our past experience, we ported a large system from Java to Go, which greatly decreased its memory footprint and garbage collection overhead.

When talking about Go:

> What remains to be seen, of course, is how all of this Go code will perform.

Yet: The JVM's GC is superior to Go's, Java tends to outperform Go, and a large memory footprint is a necessary evil for getting good performance out of any GC.

> Modern C++ ..., and would have been much more productive than go.

This is definitely not correct. Any GC language with even the most basic packaging support (go get etc.) is miles ahead of C++ productivity wise.

Why does package support make it more productive than c++? For most c++ libraries you download the source, build it (platform dependent) and set includes/libs and youre done. More so, you only tend to mess with dependencies infrequently, so who cares if it takes an extra 5 minutes once every 6 months?

Depends on the problem at hand - if youre talking about code that is not performance / memory sensitive then sure.

I agree. I may be wrong, but I don't think there's anything intrinsically better about Go vs Java from a GC perspective. Go has a developer-accessible stack, so there may be more opportunity for tuning there, but they both do escape analysis and Java will put things on the stack for you if it determines that it can. They both have loads of really smart people working on GC, so I'd be really surprised if one was far better in that regard than the other.

"I may be wrong, but I don't think there's anything intrinsically better about Go vs Java from a GC perspective."

Go has a lot more things that can be value types than Java at the moment, which can help for some use cases, and doesn't do squat for others. I'd say it's easier to use Go in a way that lets you avoid the GC entirely, but only easier, not "easy", and when you are forced on to the GC Java will of course have the advantage, at least if you have enough RAM.

It's more nuanced than "Java is always better"... which is to some extent precisely why Java is last I knew looking at adding value types, which would tip this discussion firmly back in Java's camp from a technical point of view, probably. (At least once it matured.) I'm not sure what the current status of that discussion is; all the discussions I can dig up in Google are still saying it may go in to Java 9. Anyone with a more up-to-date link, please let me know.

"Perhaps most telling that Go is a good fit is that a lack of previous exposure to the language has not been a barrier for contributors"

This. It's true at the 100% Go backend company I work at that's never hired anyone with Go experience, and I've heard many others repeat this mantra.

While many people like to think of Go's lack of features in the realms of packaging, types, syntactic sugar, etc. as downsides, they're huge upsides when learning the language. There's just so much less to learn. Onboarding a new Go developer takes days, not weeks or months.

>>>>> Java’s known performance issues made it unappealing

I have read this many times and every time I read it and it seems to me that this is just a rationalization, probably an unconscious one:

"Offering false or inauthentic excuses for our claim because we know the real reasons are much less persuasive or more embarrassing to share, or more harsh than the manufactured ones given." [1]

Maybe not the case in this particular instance, but every reason I have heard about choosing Go can be summarized as: "We chose Go because it is new new cool language. Ahh, and you can deploy by copying a single exe file"

[1] http://www.logicallyfallacious.com/index.php/logical-fallaci...

In case anyone's interested in trying CockroachDB, I have been writing some docs on how to install it, use the SQL commands, etc. So far, I've found it's easy to get started with if you're familiar with another relational DB, although its definitely still (pre)alpha.

CockroachDB docs:

https://www.grockdoc.com/cockroachdb/alpha/articles

FYI: The installation docs are at the bottom of the list.

A little late, but why not Rust? Not the right libraries? I would think no gc would be one of the biggest factors

They started the project long before Rust was stable and even today the Rust ecosystem is still lacking some libraries (understandably).

The performance potential is real but not free as the productivity isn't as good.

I've often read stories about fast refactors from Python or PHP to Go. I wonder if that would translate to a reasonably fast refactor to Rust later on, if that should become necessary or desirable.

My experience is that refactoring from a garbage-collected language to a non-garbage-collected language are slow.

Rust is different enough as non-GC'ed languages go that I'd be interested in someone's experience, whenever they have it. I could easily see it being easier or harder than trying to convert to a "conventional" language. Heck, I could even see it sort of being both, where the conversion takes longer because the compiler keeps complaining but the final outcome is solid almost instantly.

I've done a port of a small amount of code (not a full project) from lisp to Rust; it really doesn't work, as Rust's ownership semantics and circular references are somewhat orthogonal.

Also lisp is in an odd position where it has influence from functional style, but at the same time expects to have mutability always available, while Rust wants mutability to be constrained to known dynamic extents.

CockroachDB engineer here. Rust would definitely be a contender if we were starting the project today (we have several rust fans on the team), but when the project was started in early 2014 it wasn't ready yet. The language was still changing too much to make such an investment in (they hadn't even removed the @ and ~ pointer syntax from the language yet).

Longtime Rust developer here. You definitely made the right choice, it would have been a pretty big risk to bet a startup on rust back in 2014 if you didn't have the freedom to keep up to date with the breaking changes. Crazy people like me had a lot fun seeing the language evolve in realtime, but it certainly would have been stress-inducing if I was betting my livelihood on it.

Now that we're stable, please let us know if you plan on experimenting with Rust. We'd love to hear how it goes (I run the SF meetups and am always looking for speakers), and what we can do to help. One of our short/medium term goals is to make some high quality bindings to other languages, so that projects like yours could slowly introduce Rust where it makes sense.

Using Go is a Google acquihire hedge. :)

I think that someone building a product on Rust is taking a very long view and has to make an investment in the development of the language or they are at the mercy of the changes and lack of libraries.

But seriously, why not Nim?

At the time it would suffer from the same problems as Golang (GC) and Rust (not stable, few libraries around).

The gist of it is, they believe they're more productive with Go than with C++ or Java. In particular they mention Go's small idiomatic footprint and its focus on simplicity and consistency, which they say make it easier+faster to pick up the language (for new developers) and write cleaner, understandable code (for all developers).

As to performance, they expect it will be comparable to C++ and better than Java. I say "expect" because they don't provide any evidence to back that claim.

PS. I often feel like I'm walking walking on eggshells when discussing computer languages on HN. For the record, I'm neither "pro-Go" nor "anti-Go." Languages are tools, not religions. Go is just a tool: https://news.ycombinator.com/item?id=10465697

Excuse me if this is a dumb question, but I've seen a lot on HN these days where people would suggest Erlang/Elixer for this kind of use case (high performance distributed). Any reason (beyond syntax preference) that this was not one of the languages considered?

I wouldn't recommend the Erlang VM for anything that needs to itself be running at the highest performance levels. The VM is not very fast when it comes to computation. It's a great VM for what it does, and absolutely should be considered for many other such tasks, but probably not a great choice for a database.

To some extent, but a lesser extent, the same could be said for Go: http://benchmarksgame.alioth.debian.org/u64q/which-programs-... (It's a bit weird reading that page, because it's broken into two graphs, but they appear to be about the same thing.)

If I were thinking "trendy language to write my next database in" personally I'd favor Rust by a country mile. This is exactly where you'd like all that stuff Rust provides just for correctness' sake, and this is the sort of task where you may even be able to outrun C++ if Rust helps you write correct code with fewer barriers slathered about for correctness.

Edit: Ha, and just below this comment as I read it I see the team commenting about Rust.

Riak is a database written in Erlang.

Erlang is not good for "high performance distributed" systems. It's good for systems whose primary characteristic is uptime rather than latency or throughput.

Can anyone link a reference for known performance issues in Java? And is it issues in Java or the JVM?

I can't provide a direct link. But generally Java itself is rarely the problem. The language is fine, the byte code is great.

The JVM on the other hand with its stop the world garbage collector is a constant source of performance issues. Especially once you start getting over 2-4GB heaps. GC cycles can take 1s+

The Golang GC is "better" than the JVM GC, according to this blog post?

Though I think it's suggesting that the Golang GC is easier to tune and configure

There are plenty of options for storing select data off heap, which a database such as Cockroach could certainly have taken advantage of.

so this is an oversight on their part? it sounds like they had maybe a half-baked justification, but idk

I agree to most points. One thing I struggled trying it is lack of information on interenet as compared to more mature languages like c++ or java. Since most of the cockroachdb team including founders come from google, I wonder if they have access to better advice too.

Reply on news.ycombinator.com