Hacker News Reader

Comment by TekMol | Hacker News Reader

    it's not very realistic

It is totally realistic. I have multiple projects running that do not use any external code. With millions of users.

Those projects are PHP though. PHP has a lot of bells and whistles on board. Unfortunately with PHP 8, they started introducing a lot of breaking changes. So they lost my trust and I will use Python for new projects.

Not sure yet if I will use Django (As you said, it seems pretty stable) or do the good old CGI approach. If I do the latter, I will probably write a new CGI library for it, because in the Python ecosystem it is all about long running processes nowadays and I think that is the wrong approach.

View on HN

There are some good reasons that everything, PHP and Python included, moved away from CGI to long-running processes.

Name one.

In general, startup takes time - import dependencies, read config files, make a database connection, and so on. That time can be noticeable.

Years ago I had to deploy on a network file system which was very slow for file stats. (It was designed for HPC and bandwidth.)

Python startup (by default) requires a large number of stat calls. It took our CGI app over a second just to start.

I was able to improve it by packaging the Python std lib as a zipfile, and our app as another, since zipimports avoid a lot of stat calls.

TurboGears had just come out. I did a prototype using it. Its was so much faster!

For organizational reasons were were not able to switch. IT took over and, after several years, replaced it all with a Java version.

What is your problem with startup time? Have you timed it? How long does it take?

PHP starts up from scratch on every request, yet it blows Python out of the water performance wise.

It's been a while since I've seen anyone brag about PHP's speed.

PHP is quite slow at encoding and decoding JSON, very slow at any kind of tree data structure, and has among the highest memory usage of any language at printing a string of text to a console or otherwise.

Python is faster at all of those things and it's quite slow. Those are kind of web related tasks, and the web benchmarks for say django vs laravel don't show that PHP is any kind of winner.

If he think's CGI startup time is acceptable then I guess maybe PHP seems fast then?

My bigger issue with the CGI model is the lack of handling connections as streaming. You can't parse a file while it's still being uploaded, and that can eat up a lot of memory on larger files.

I know startup times can take a toll like establishing db connections, cache checking (like redis), file imports, etc. If you are using something that starts up for every request, then you have to be aware of those kinds of things and design around them.

By the time everything is imported and running you can be into the territory of really not wanting to do it on every request. I’ve not timed it in my case for a while but you’re definitely looking at 100s or 1000s of ms.

> Have you timed it?

I literally gave a timing number in my comment.

The >1 second time was for a very unusual circumstance. Unless you timed PHP startup time using straight CGI (not mod_cgi, nor FastCGI) on a Lustre file system about 15 years ago, with the appropriate set of extensions enabled for what our application needed, you can't really compare the two.

> PHP starts up from scratch on every request

Congratulations.

Now getting back to your request to "Name one", the PHP documentation at https://www.php.net/manual/en/install.unix.commandline.php gives essentially the same answer as I did:

"By default, PHP is built as both a CLI and CGI program, which can be used for CGI processing. If you are running a web server that PHP has module support for, you should generally go for that solution for performance reasons."

Even to execute the most minimal binary fork/execve takes a not insignificant amount of time. Thats one of the reasons why CGI fell by the wayside.

Perf

Long running processes only need to load their imports, configuration, database connections, etc. once, and don't incur overhead for each request. On top of this, things like imports get to be cached, and there's less need for a database connection pooler to broker connections.

Particularly async runtimes have become more popular - where you can have a single process handling concurrent requests and awaiting/yielding in between their api/db calls. Instead of having many processes that don't share resources that might be waiting 10ms/100ms for api/db calls.

For applications that make large numbers of api/db calls - especially slower ones, may prefer async runtimes over individual processes per request.

I'm not at all saying CGI is a bad option, just that there are real benefits to reusing, pooling and caching resources.

Long-running processes have their uses but they are not the only tool or the end of the evolution. "Serverless lambda" is one example of how moving back to one-off processes provides a different kind of value.

Reply on news.ycombinator.com