Why I Left IBM to Work on CockroachDB | Hacker News Reader

Why I Left IBM to Work on CockroachDB

cockroachlabs.com

141 pointsorangechairs90 comments

Seems like younger tech companies finally realized IBM was full of mid to high end talent that is severely underpaid. For instance Facebook and Amazon have poached a lot of the Linux kernel talent that used to work on POWER arch and performance. Although the products tend to be interesting, seeing the sheer fame and popularity younger companies and contributors are enjoying must be a downer. It's bittersweet because a well run IBM would still deliver a lot more interesting projects than most startups that are hanging on for dear life until the big event like an IPO or acquisition.

I used to be in a trio of "hackers" as a kid. We were the ones who were always learning new stuff about programming. We picked up javascript before anyone else in our community at an early age, and we also explored Digit magazine CDs and discovered Ubuntu earlier. This was a time when most people didn't even have an internet connection (oddly me included) The two others who were with me were better at it than I was. One is at Broadcom, if you have a Samsung phone, your Bluetooth works because of her. Quite literally. She is the best at the drivers. The other is in some silo at IBM. He's been there forever, and he says he's found his peace there.

These two wrote their own music player in VB during high school, called it FireAMP. They used to do things that I couldn't understand. Right now, neither of them is even writing a blog. They're in their silos and I'm praying someone gets them out and helps them share their knowledge with the world.

They're the best programmers I know, and I've been at several conferences and spoken at a few. I wish they'd not sunk into their corporate holes so hard.

I’ve rejected Facebook and my friend rejected Google offers.

We both found something we like more than chasing other peoples dreams, your friends are probably the same.

They very likely are sharing their knowledge, just not for free, publicly, and not at the expense of their personal lives.

I have heard the same from a manager I know at Oracle. Not to say there’s a bunch of hidden talent there, but in the last couple years there has been a long string of good, high level people who got huge offers from FAANG and walked away. Oracle doesn’t even try to compete, and couldn’t. In most cases that person is also replaced by an offshore, sub-par employee with no money to hire a real replacement.

This seems to be a recurring pattern. Should we be worried? What if FAANG just poach all the good talents and put Intel, IBM, Oracle, etc out of business? Is this not anticompetitive?

Competition implies that there are winners and losers. If FAANG just poaches all the talent, then Intel, IBM and Oracle will just have to start competing for the talent they need instead of resting on their laurels.

Anti-competitive would be if they agreed NOT to poach talent in order to keep salaries low.

As far as 'autonomic' DB tuning, it'd be nice to essentially have a model that provides a set query performance and redundancy for a lowest possible dollar amount for a given workload. i.e I want my application's queries to have 99th percentile <=100ms latency - use whatever combination of memory, flash, disk, or compute is necessary to make that happen with four-nines reliability minimizing price. It'd then figure out what columns need to be stored on which medium, what in-memory indexes should exist, how many 'nodes' should be spot-instances vs reserved and so on..

Might I suggest we add a touch of subtlety and call it “RoachDB”. Still the that undying stoic of the much disliked critter, but without invoking the full image of a cockroach. Love the work you guys are doing <3

A case could be made if you consider that the first part of the word is masculine and inappropriate.

Some references to the mentioned works, missing from the OP:

DB2 BLU: https://researcher.watson.ibm.com/researcher/files/us-ipandi...

Wildfire "v0", the "failed" HTAP project: https://researcher.watson.ibm.com/researcher/files/us-ytian/...

Wildfire "v1", the shared-nothing architecture over cloud storage (EventStore 1.0?): http://cidrdb.org/cidr2017/papers/p123-barber-cidr17.pdf

DB2 EventStore (EventStore 2.0?): http://www.vldb.org/pvldb/vol13/p3299-garcia-arellano.pdf

Have to say that whole exp is extremely fascinating to a non programmer such as me. Looks like once you dig deep into system programming you can get yourself some challenging and comfortable long term job.

Stick to the fundamental bro.

What is the purpose of this post apart from self promotion of its author?

It looks like some sort of a "textual CV", where the author tries to push the agenda that he is not only a technical expert (without any real examples, just buzzwords), but also a managerial expert (here even less examples, just sucking own dick).

We can learn that he worked "really hard" on IBM products, so hard that he fell asleep during work (dear recruiters I can work hard at a startup, pleas hire me!!!), yet those products seem to be so irrelevant, that once when he writes about cloud, he does not even have the courtesy to list IBM as a cloud provider.

Also the integrity of this piece: guy writes about others changing their jobs to maximize the salary, but of course at the time when the author switched job, he didnt do it for better money, he did it for the great product and great team :)

IBM is a sinking ship, so for me it looks just like a story of a guy who changed job to earn more money, which for some reason allowed him to put his CV on their own website, so he can be headhunted by someone else (who will pay him even more money?).

Disagree hard. This piece is interesting because it is a relatable window into the history of database concepts and what it's like being on teams that work on them. For anyone who has done serious work on databases its a fascinating read. For example, IBM was working on a cloud native analytical database that separated storage from compute!? They could have had the next snowflake on their hands.

Surprised the Author didn't mention IBM Canada being a very unstable place to work.

Eh? Source?

While the article was interesting, I found it quite odd that IBM would put someone fresh out of university in a team lead position of an area they initially had no interest in, and then make them team lead again of an area they admittedly knew nothing about. Maybe his teams were all India based?

I believe the open systems db2 team was mainly based out of Canada, but I don't think you were really honestly asking that question.

However, in case I can share a bit that I may know some small amount about is that the area that the author worked on is quite well-known to IBMs db2 customers. The ability for db2 to self-tune has put it ahead of sql server, oracle and others in some ways, for decades. While the author clearly got hired and found himself in a good niche at IBM and grew there, IBMs database teams have definitely hired and sustained people from various backgrounds because the work can be challenging and rewarding.

I'm saying this because one of my uncles maintained the lock manager in the mainframe db2 codebase for most of a couple of decades including being there at the time the author mentioned, working on sysplex and parallel sysplex, and he entered that gig without a particular interest in databases as he told it. But he really did take to assembly and production code maintenance

> I don't think you were really honestly asking that question.

I don't know what you're getting at by this statement.

I'm asking how it comes to be that someone fresh out of uni is immediately made a Team Lead. It doesn't tally with my understanding of career ladders.

Sometimes these things happen quite randomly. You'd think that a big company like IBM would have a long line of people ready to take on a juicy project like this, but at the time, all of those people were tied up with more pressing work (customer commits, a high-profile performance project, etc).

Also, I've clearly glossed over a bunch of details but I wasn't hired as a team lead. That being said when management saw that I was capable of driving the team (mostly) on my own, they progressively stepped back further and further until I was essentially running the team. This was over the course of a few years, which I've compressed in the writeup.

All that to say that career ladders aren't fixed, and if you find yourself in the right place at the right time, you can bypass a lot of the traditional ladder climbing.

Thanks for clarifying. I was curious, not critical.

> I don't know what you're getting at by this statement.

Saying "Maybe his teams were all India based?" sounds dismissive or pejorative, as though you were saying that he must have been somewhere that was not doing premier work, or working as part of a cooperative team in a way that valued the work he and his team did.

However the essay clearly described how innovative his ongoing work and contributions were when they were being done, including linking to research papers. They were progressively pushing forward the state of the art in the database world, and did it in production in wide deployment. What happened was that the culture at IBM identified him as someone who could lead, and as I described, their database organizations at the time helped grow people into these roles.

That's quite an interpretation. What I was suggesting was that maybe he was the Canadian face of a remote team.

twinkieDB could survive an apocalypse. much better name

Maybe not: https://www.npr.org/2020/10/15/923411578/a-disturbing-twinki...

Well even in that article they say it was most likely mold that got in there before packaging.

Hence the "maybe"...

Cockroach employee here, and I'm all in on TwinkieDB. Please start a write-in campaign.

Seems likely TwinkieDB would likely cause some legal problems.

I really hate that name. I'm sorry, I know it's been said a million times before.

I've thought about forking it and just running a script to rename it to "ButterflyDb" or something pleasant.

Nobody thinks "cockroach = doesn't die". We think "disease" because we evolved to have a strong rejection response to these creepy crawlies. Roaches cause disease. Does anybody want that with their infra?

They could sell this so hard if they named it "ice cream" or "cheesecake" or "coffee" or anything but slimy cock roach.

> Nobody thinks "cockroach = doesn't die". We think "disease" because we evolved to have a strong rejection response to these creepy crawlies. Roaches cause disease. Does anybody want that with their infra?

The problem with statements like this is that's exactly what I think. I hear that name and immediately think of a database that's very resilient.

If you want to win the argument about changing the name, don't start by claiming that people who think differently than you don't exist.

I don't think his argument is "this name has cost you every customer," just "this name has cost you SOME customers." So I think his argument about the name is stronger than yours - at my current job the Cockroach marketing emails we get are treated as much more of a joke than the ones from their competition, and it's solely because of the name.

> I don't think his argument is "this name has cost you every customer," just "this name has cost you SOME customers."

Likewise, I don't think my argument has anything to do with lost customers whatsoever.

I am merely criticizing the argumentative technique of claiming that one's position is so unassailable that people who disagree simply don't exist.

I have no attachment to the Cockroach name or product. The only horse I have in the race at all is that I hope the project changes the name just so I don't have to read the same debates every single time the database hits the HN front page.

Do you think that because your first thought of "cockroach" is "doesn't die," or because you've researched about this DB in particular?

I have never used or researched CockroachDB, my only exposure to it is when every few months a story about it comes to the front page of HN and the top thread is inevitably about people being upset at the name.

I remember reading the first thread and being confused why so many people were missing the "obvious" meaning behind the name. Clearly it's not as obvious as I had initially thought, but it's the first thing that came to my mind.

I know that's not the answer you want to hear, and I apologize for that.

IMO this is a stronger reason - it's not that nobody gets the intended meaning, it's that enough people get the unintended one and every HN thread is derailed. If that's the sentiment on a community with downvotes and a strong culture of "we just care about the technical issues", imagine the sentiment on a team considering adopting it.

FWIW I will never proactively advocate for this DB or try it out because I don't want other coworkers to feel squicked. I'll do my job if I find myself on a team using it, of course, but in my many years working on software, I've found that you can't make the computers happy unless the people typing into the computers are happy. Limiting the pool of people who can be happy reading CockroachDB docs when they're paged at 2am for an outage seems not worth the chance that it might page us at 2am slightly less.

Not the OP but the first time I heard “cockroach db” I assumed resiliency because of the childhood myth/meme that in the event of nuclear winter the cockroaches would survive.

This is exactly what I pictured in my head when I first read the name.

I'd be interested to know how many of the people who are upset about the name have heard and remember that theory.

I remember the urban legend meme that cockroaches could survive anything, even nuclear war.

I also used to live somewhere that we’d occasionally see a cockroach, and they’re disgusting. Any time I hear the word cockroach, immediately it reminds me of the times I was going to the bathroom and looked up to see a cockroach, jumping back in surprise. It’s a much stronger emotion attached to the word. Just immediate feeling of disgust.

I wouldn’t mind working on a project that uses cockroachDB if it really is the best tool for the job, but I do feel a little weird about it. And the name has kept me from really looking into it previously...

I did too! That's not mutually exclusive with visceral, extrarational revulsion at the name.

My previous (imho, thorough) takedown on whether criticizing Cockroach's name is "bikeshedding":

https://news.ycombinator.com/item?id=14310444

First time I've ever heard of it was right now, and my first thought was that it was named for resilience. Only learned that was the intention reading your comment.

ResilientDB would practically sell itself!

BulletproofDB

CoronaDB

Despite everything that's happened, that still makes me first think of the Mexican beer.

What about CoviDB-20?

As somebody who hadn't heard of this DB before, my thought pattern was basically:

"cockroachDB? Oh is this a joke project...? Why would they leave IBM for a...oh, cockroach like can survive a nuclear explosion. Huh."

I like it. It's memorable, and for me it does evoke "doesn't die". I'd even argue that, out of all the DBs, the name is one of the things that made it successful... even if you don't like it, you're gonna remember it.

The best part about a name like CockroachDB is it keeps all the people away who would complain about a database being named CockroachDB, who would most likely be your lowest value, highest complaining customers.

I don’t think this is true. It feels like the same kind of argument as “one who cares about their suitor’s appearance must not care about the stuff that really matters, their personality.” It’s not true and it’s a false dichotomy.

Software projects that use databases often have lots of people working on it, reporting to people who don’t work on it. While I personally believe it’s easy enough to bury “CockroachDB” from sight, it may not be so easy everywhere.

Lots of people, techies and not, care about aesthetics and appeal. It’s not rational (in any technological sense), but it is what it is.

I am not presenting a dichotomy. I am not saying you must either care about aesthetics and appeal or function and form.

I am saying a good technologist can look past aesthetics or appeal, which are superficial characteristics in my opinion, and judge a technology based on its technical merits and the problems it is capable of solving.

Yep. The data will be allowed to become inconsistent garbage but the DB engine will still be running in a 100% CPU utilisation loop - just like a headless cockroach running in circles on the floor.

Agreed. I grew up in a roach-infested apartment and hearing the name as an adult gives me creeps.

A counterpoint: my home has a roach infestation and I killed two today and... I must say I love this name.

When I think about cockroaches the first thing that comes to mind is that they will surely outlive us. I don't want my data storage to be pretty, I want it to survive hardware failures and outlast pretty much everything.

But so does "IBM", for computer professionals who came of age in a certain era…

Oh really? I'm still quite young so I may not have heard of that. Curious to learn more tho.

There used to be a somewhat common phrase "nobody gets fired for buying IBM", implying that, while less than good, their products were at least a safe choice.

Maybe it's just the changing of the generations, or proliferation of SaaS and AWS etc, but I haven't heard the phrase used in probably 10 or 15 years. That in itself is probably an indication of where they are at more than anything.

Edit: Safe choice for the purchasing manager. The poor sods who had to deal with it, on the other hand...

Those of us who entered the field in the 1980s expected to face a future of writing COBOL for some IBM compatible computer, wearing a tie.

Whatever hardships life had in store for me, I'm grateful that none of these three came to pass.

I've encountered plenty of sysadmins who either wince or get visibly angry at the mere mention of "Lotus Notes"

Notes is actually a really cool idea. A cross-platform rapid application development environment with a built-in replicated document-oriented database. I guess it is just a pity that there were some flaws in the execution, plus it got shoehorned as an email system rather than emphasising its full potential.

It's mostly a joke. Mostly...

I also think that in the day of corporate data inspection including a not safe for work term in your application (even if it's a sub-phrase) isn't going to help with adoption at all - some potential customers may simply be blocked from searching for it (or flagged to HR) due to bad regexes.

If you find yourself working at a place where it's frowned on to use words like cocktail, cocky or cockroach.... then it might be time to find somewhere less stultifying.

Absolutely - but some people do work in such places and why don't you want those people to be able to use your tool?

Meh. I’m likely as creeped by the crawlies as you... still not really phased by the name. I can imagine some business stakeholders reacting negatively though - but they might write that off as general tech weirdness...

I think the problem is the morpheme "roach", which evokes uncleanliness.

I would propose renaming it to "CockBase" or, even better, "CockDB".

Don't @ me lest it is to figure out where to send the naming royalties chex.

Obvious innuendo aside, this evokes getting "cock-a-doodle-dooed" by my alerting / incident response software because it's blowing up somehow.

probably be good as the backend for hotdog / not-hotdog

I hate(fear?) cockroaches with the entirety of my being, especially the flying variant. As a child, a cockroach once bit my toe. That's one times too many. I immediately grab the nearest hard object I can find to slay these F#$%^ERS. I've also heard stories of cockroaches entering people's ears (EWW!).

That said, I have no problem with the name CockroachDB whatsoever.

> I know it's been said a million times before.

But yet this a comment about CRDB's name finds it way to the top of the HN discussion once again.

What does not die is the need to complain about a product name instead of a constructive discussion about what it actually does.

The fact that this is a purely technical community proves the name is bad imo. Managers who aren’t tech savvy won’t adopt it.

If you work with management that base technical decisions on a single word name, one with relevance to its ability, then you have bigger problems than database platform names.

"What is this Postgre thing? I can't pronounce it so it can't be any good."

"MySQL sounds like a toy. We need 'OurSQL'."

"Oracle is the right choice because I'm familiar with the name."

"Nobody ever got fired for buying IBM."

"Eww, bugs! Gross!"

You're in luck - there's a database just for you! It's all of CockroachDB, with whatever name you want:

https://github.com/tbg/bikesheddb

Well played, but I don't think naming an enterprise product is bikeshedding.

Agreed, and I pretty thoroughly unpacked that argument in the thread from three years ago:

https://news.ycombinator.com/item?id=14309984

https://news.ycombinator.com/item?id=14310444

You are right. Honestly, it makes me severely question the judgment of the leadership. I can just imagine the kind of difficult tech personality heading such an operation that refuses to change the name. All of the replies disagreeing with you are filed in my "engineers aren't socially normal" mental folder.

Agreed, first impression matter. We evaluate it by name before we even know it, in milliseconds. It leaves a bad taste.

I have a phobia of cockroaches, having grown up in a place where they would appear here and there. I am grossed out by their body's front side especially. I've avoided opening their site in case they put up an actual picture of a cockroach.

TwinkieDB

Good thing we don't pick our tech stack just based on how the name sounds... right?

nervous laughing

Actually I used to hate it but now I'm starting to like it.

I have to agree with this , Phoenix DB maybe ?

Once you need to pitch it to your boss she might think your joking or not taking it seriously.

I would be reluctant to use this in any work projects

Unfortunately Tardigrade DB has already been taken. Radiodurans DB might be a good alternative.

Surprised you didn't mention the more salient implication: cockroaches are bugs.

I don't mind the insect reference at all. But my problem is, I read Cockroach and right away I think this is some service on top of Kafka so now I think to use this CockroachDB I have to deploy a Kafka infra too and I just move on.

There are plenty of unpleasant names in tech. I don’t think this is going to make a difference.

GIMP and git come to mind. What else?

Master/slave. Male/female. Dongle.

These having been issues for some people. Myself not included.

Hopefully, once they get acquired, they'll be forced to change it

Reply on news.ycombinator.com