cgi as a protocol, really a calling convention for launching processes, make sense, it fits a specific niche in the ecosystem. fastcgi does not, fastcgi is a different incompatible http, that is, fastcgi does nothing that http does better. Did we all just collectively loose our ability to think critically, locked on to the fact that the common usage was that a http server launches a cgi process and when the time came to make that process it's own service, said we need cgi for services, creating fastcgi, forgetting that http already works just fine as a service?
I am not really a back-end programmer, I am a sys-admin who sometimes makes web-based tooling. it is very possible there is a subtlety to this I missed. But I was a lot happier when I gave up on fastcgi and just made each service a http server with a reverse proxy in front to dispatch the requests.
> do have an issue with programs that have a lot of startup overhead (e.g. things written in Python
Unless you have a lot of startups its probably not a problem, I would have thought.
If you wrote your service in C like god intended(sarcasm) this is not a problem, unix systems are traditionally by design very good at starting processes.
However python(my favorite language for what it's worth) has a lot of baggage it needs to sort out when it starts. so python specifically and any interpreted language that takes more than a few milliseconds to start in general starts to suffer under heavy loads in the one process per request model.
Thus the motivation to make it one process for many requests.
The figures from 1990 are not the same as the figures from 2024. "heavy" here is so high it is not a realistic problem for 80% of sites.
On current hardware it can serve up to a few hundreds requests/s without too much trouble.
There's also the trick of pre starting a pool of processes beforehand and handing the data to them when it comes. It is not implemented in listen yet, but would not be too hard to do.
This was true of cgi scripts written in perl too, until they made the frontpage of slashdot.