Hacker News Reader

Comment by swe_dima | Hacker News Reader

My concern is number of database connections. In the example it's 100 fibers per worker, at that rate you are going to exhaust db connections sooner. Happy to be wrong.

View on HN

Carmine (which coded this Fibers update) wrote about this in his blog. See the section 'The database connection math'. And no, you won't have one connection per fiber.

The difference is staggering when you compare to threaded mode: it requires 1,320 database connections to run the same benchmark that the fiber mode runs with 60.

https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/

Pretty sure that is entirely an unfair comparison. You don't need a connection per thread either. It is common to have web servers handles thousands of requests per second with a connection pool size of 20. As long as the handler threads borrow a connection for only a little time and block waiting for a connection while other threads are doing their thing, it works out fine.

There is absolutely no logical reason why database pool sizing could be a reason for preferring fibers over threads.

I think you want to make sure you don't hold a database connection while waiting on other slow async work (like outgoing HTTP requests). Then you can more feasibly have more workers than pool size. It's just very tricky to do this in Rails...

I don't know Solid Queue or the rails environment, but I expect that not every worker will create it's own connection, there should be a connection pool in-between

I think by default they check out a new connection when obtained a job and then release it. In comparison with some async languages like JS where a connection is only checked when a query is about to be executed

this used to be the case in Rails, but hasn't been for at least a year. Nowadays each framework-controlled action (like record.save) checks out a connection and then returns it back; this was implemented as part of the whole "run rails in fiber-based server" push

Reply on news.ycombinator.com