FOQS: Making a distributed priority queue disaster-ready | Hacker News Reader

FOQS: Making a distributed priority queue disaster-ready

engineering.fb.com

55 pointsel_duderino13 comments

You can already get almost the exact same functionality via Vitess Messaging, also built on MySQL and HA across regions, which has been tested at YouTube scale. Vitess doesn't support ordered messages, but otherwise the architecture is almost identical.

https://vitess.io/docs/12.0/reference/features/messaging/

Sounds cool, almost like competition for Kafka if it solves global multi-DC replication and has a scaling story. Is FB / “MarkZ The Generous”* planning to open-source this valuable sounding thing? I wasn’t able to find a mention about sharing it publicly in TFA.

Would be a nice gesture, given how much $FB has benefited from running the entirety of the worlds largest social graph on top of MySQL.

* lol

> given how much $FB has benefited from running the entirety of the worlds largest social graph on top of MySQL.

They've contributed quite a lot back to the MySQL community, I thought? MyRocks comes to mind

http://myrocks.io/

Has BookFace contributed back commensurately to the degree Señor Mark Fuckerberg has benefitted? That would be truly impressive and challenging, to the tune of tens or hundreds of billions of dollars.

MySQL is truly the unsung hero of FB, the whole operation would not have taken off and become a smash hit in the first place without riding atop the back good ol' MySQL.

Unfortunately, Postgres wasn't such hot shit 15 years ago. Today Postgres is AMAZING, it is truly a great day to be alive and a programmer. Now our best choices for relational DBs are Postgres, MySQL, or Oracle 21c (lol, Oracle <3's $$$ but at least Larry is straightforward and honest about it :)

The same argument can be applied to Linux and apply to virtually all companies having a backend?

What does this change? Few companies have the resources making it even potentially feasible that they even could give back at the scale Faebook could.

It's a two-way street. Facebook has made significant contributions to MySQL, including commits (they're literally one of the top third-party contributors), bug reports, and overall roadmap. Several of FB's top DB engineers previously worked for MySQL AB, Sun, and/or Oracle.

> Would be a nice gesture, given how much $FB has benefited from running the entirety of the worlds largest social graph on top of MySQL.

There are plenty things to shit on Facebook for, but "not contributing to open source" is definitely not one of them. React, GraphQL, Cassandra, etc. are all hugely successful and influential projects.

>Is FB / “MarkZ The Generous” planning to open-source this valuable sounding thing?*

FB usually open sources many projects - but I wouldn't be surprised if this never leaves FB just on practicality. Anytime there's "global data-center scale" state management thingy, it's usually deeply integrated with whatever flavor of Zookeeper/Chubby/Service Discovery/AWS/Global metadata store/GPS Clock/Intern's cron job that makes it a pain to decouple, even worse to actually test against an open source version of these stacks.

I mentioned this in a top level comment, but YouTube already open sourced their version of it in Vitess, with almost identical characteristics.

It's interesting that Facebook has created another queuing system with priority rules. I believe beanstalkd originated as a result of Facebook's use cases.

Seems like a really cool piece of tech.

Beanstalkd was created by a company that ran an early third-party Facebook app (Causes), not Facebook itself.

Facebook used their in-house "async" system, described in the post, for many years. Prior to that (over a decade ago) I believe they used Gearman for some things, but I'm not certain, might be misremembering. I don't know if they ever used beanstalkd early on.

Cool! I didn't know this!

Reply on news.ycombinator.com