* We implemented a feature to share memory between these connection handling processes so that makes it a bit more efficient
* Longer term we are thinking of switching to a thread based model. This is how our other API YCQL works.
(cto / co-founder)
DynamoDB/ElasticSearch's approach of not needing to establish/maintain a connection seems like the ideal solution. I think for applications that would utilize your DB, a minor hit to the speed of an individual request is well worth it as compared to being able to scale up the amount of simultaneous requests.
Peeling the onion one more layer, there are two underlying features that are required:
1) Move to a threaded model to be able to scale instantaneously. This is already the case with YCQL, we're planning on doing this for YSQL.
2) There needs to be a change to the client drivers in order to become smarter to deal with how the app connects to the DB. In fact, we're working on enabling this for the Spring (Java) ecosystem by enhancing the JDSB driver (still in the early stages): https://github.com/YugaByte/ybjdbc
(cto/co-founder)
* The node given to the JDBC driver could be down, or nodes could get added/removed over time. This makes "discovery of nodes" a problem.
* We could use a random round robin strategy to connect to nodes - where client connects to a random cluster node which internally connects to the appropriate node. This would result in an increased latency, and also an increase in the net number of connections needed.
These may not matter if the load balancer is smart like in the case of Kubernetes (and where the extra hop becomes mandatory as well). But for non-k8s deployments, these help.