https://code.facebook.com/posts/1007323976059780/continuous-...
https://code.facebook.com/posts/1007323976059780/continuous-...
I want to feel bad for GibLab, but it's really, really hard when they hire people like that.
I understand that there are perfectly legitimate reasons for using a database as a queue. If you frequently need to look at and rearrange your jobs while they're in flight, chances are a pure FIFO structure probably doesn't work that well for you anyways. If the enqueue is contingent on a transaction committing, probably makes a lot of sense for the job to be in the same database. You don't need to tell me -- I've seen more than a few in production.
But an actual message queue also "works perfectly fine" based on the information provided, and I would imagine that a company like Facebook probably already has a few of those lying around. It would have been a conscious choice to use MySQL as a queue.
I swear I'm actually, seriously just curious!
We use it because it works well for us. We've put a lot of work into making MySQL scale for us to the point where it's a very well supported system and one of the main choices for a lot of storage decisions.
We even use MySQL as a queue for Facebook Messenger. More details about this:
https://www.youtube.com/watch?v=eADBCKKf8PA
https://code.facebook.com/posts/820258981365363/building-mob...
https://githubengineering.com/moving-persistent-data-out-of-...