Hacker News Reader

Also -- if you look at the infra updated from the linked article, they mention something about 3M updates/hour to a pg table ([1], slide 9) triggering continuous vacuums. This feels like using a db table as a queue which is not going to be fun at moderate to high loads.

[1] https://about.gitlab.com/2016/09/26/infrastructure-update/

View on HN

It's not updates, it's just querying, updates are not that bad, the main issue there was that the CI runner was keeping a lock in the database while going for the filesystem to get the commit, this generated a lot of contention.

Still this is something we need to fix in our CI implementation because, as you say, databases are not good queueing systems.

For sure there is much we can improve to reduce DB updates, but we do use Redis for queues.

Reply on news.ycombinator.com