I would be really curious to know what the overhead of actually creating the connection and loading the collections on each request would be. Is this anything more than a micro-optimization?
Though the overhead can be rather minimal, it still exists:
- Connect to the database
- Query the Collection
- Make a request
- Close the connection
Also, these four separate request are asynchronous, which means you could end up with X web requests having connected to the DB but not having a collection, Y requests having a collection and N requests waiting to be closed.
This is not a real problem (as long as you don't have 4 servers connecting to the same Db with 65 thousand connections each :-) ) but having multiple CONCURRENT connections to the same server is pretty useless in a SINGLE-THREADED context like a Node.js app.
Now, if you were to use PHP, that's a different matter...
With that said, you shouldn't create a connection per request, you should use a connection pool.
You may be thinking you can fire up multiple request at the same time (ie: fire the requests, let the I/O do the job, get all your requests back) but since you are in a single-threaded environment, you do exactly the same as I/O on a single connection (plus the overhead of opening new sockets and establishing hands shaking with the server).
And yes, that is exactly what I am thinking. More database connections = more concurrent database requests. I don't understand your reasoning about being in a single threaded environment. You made a statement but you haven't explained it.
From what I can see, this problem is simply serial vs parallel. Is there something I'm missing? I don't see how the fact that node.js is single threaded changes the situation since the database requests aren't working within the node.js context.
Edit: sorry, my previous reply may have been confusing. I was responding to your comment that having multiple connections is useless in a single-threaded environment. I was not suggesting that it may be helpful for the OP to make a new connection for each incoming http request.
In both cases, the same process of reading/writing to one or more connection sockets happen, you send and receive the same content to MongoDb: you can not send twice as much queries, you're gonna send a query, THEN you're gonna send another, then you get an answer THEN you get another answer. Everything happens in the same sequence.
Also take in account MongoDb's own concurrency properties.
The balance is between how big your pool is and how many messages are in flight. In the future this will change as MongoDB moves to some sort of eventloop or multiplexing of threads to sockets. But for the moment you have to strike a balance between operations in flight and how many sockets you open on the server.
Another thing to keep in mind is that the more socket connections you open on the mongodb server the more memory needs to allocated on the heap and the more contexts switches between threads will happen.