Uber's Move Away from PostgreSQL

It has been pointed out before, but I'll mention again. It is impressive how PostreSQL community reacted. There was no immature name calling, no blaming users. They analyze the problem, mention possible solutions, admit known shortcomings and so on. This is a community worth being a part of and engineers worth working with.

Changing database platforms is a big project, and not something you do without consulting experts first.

More than anything, I'm shocked that Uber eng mgmt didn't reach out to the postgres core team for help first -- as you can see, they'd gladly have helped.

So far is very cool how dramas were avoided regarding to this switch. I think that even more "corporate" entities should switch to acting like Postgres folks, taking it well when people move away from their systems. Sometimes moving away from a system is not necessarily just a technological matter: people get tired of how a given system is designed, of its complexities or on the contrary of its simplicity that does not hide low level details. Certain times the problem is the lack of high quality documentation and information about how to run such systems at non-obvious scale. I even saw people moving away to technologies because they were struggling with their initial stack, and a vendor of the target technology approached them saying "we can help for free if you switch to X". There are certain situations where a system that is not performing well for workload A could be tuned to work well but there is another that can just be used out of the box with success for the same workload. To avoid becoming a super expert of everything is also an option. The thing I don't understand about all this, is the propaganda made by the switchers. Sometimes you see companies almost working with PR firms of database XYZ (the target of the switch) in order to handle the communication. Or blog posts from well known companies blaming certain databases in a way that will be undetected by 95% of readers, but where the problem is, actually, a misunderstanding of how the system works (in part the Uber case qualifies apparently?). So a good rule could be: we have tons of open solutions now, use what you want, switch all the times you want, if you really care blog about your switch, but: 1) Tell about your experience in a subjective way not stressing the shortcomings you saw in your former system of choice as general problems. 2) Consider instead blogging what you are doing with your new system and why it performs well, instead of analyzing while the previous one did not work. 3) If you really want to make a comparative analysis, put weeks of work on it, so that you make the best you can in order to really explore all the tuning and options that were available in the system you are abandoning. If after the post the community will point you in the direction of errors, both correct your old post and make a new post with updates. So I think it's great to see folks with a vibe like Postres are still around, but conflicts could be avoided also by being more soft in criticizing solutions that, after all, kept your business running for months or years.

From the linked article at http://use-the-index-luke.com/blog/2016-07-29/on-ubers-choic...

"Unfortunately, the article doesn’t make this very clear because it doesn’t mention how their requirements changed with the introduction of Schemaless compared to 2013, when they migrated from MySQL to PostgreSQL."

MySQL -> PostgreSQL -> MySQL ...

Need to keep these developers busy!

But seriously, migrating databases is a huge effort from my experience, where a lot of preperation needs to take place to not corrupt any data or get data in an inconsistent state.

So this takes quite some engineering effort away from feature development.

This article strikes me as more honest/objective than http://blog.2ndquadrant.com/thoughts-on-ubers-list-of-postgr....

Uber says the moved to their "schemaless", which means, as far as I understand, that every update will always touch at least one index. Hence am not sure I understand why Robert and Markus are not clear about the HOT vs Non-HOT case; all the updates will be Non-HOT.

I've had success with bucardo, not sure if they tried that. Going to mysql will pose its own set of problems because both of these databases are not really designed for that kind of scale.