Hacker News Reader

Comment by _delirium | Hacker News Reader

It's kind of interesting that his approach, using essentially 1980s-era scaling technology (inetd forking a server per request, plus old-style CGI for dynamic content) has no trouble at all running these high-traffic websites on a tiny VPS.

edit: Though come to think of it it shouldn't be that surprising. If that architecture managed to work at all on 80s hardware, it should scream on anything from 2011, even a VPS.

View on HN

Persistent server processes are an artifact of the shift towards Python, Ruby, and other languages which optimize programming time over execution efficiency. When just starting up the binary takes over a second (to load frameworks and libraries and plugins and so on), the process-per-connection model is completely unworkable. That's why FastCGI, SCGI, WSGI, and other similar protocols all became popular at the same time as Rails and Django.

If you're willing to use pre-compiled binaries, with few or no large dependencies, then CGI becomes practical once more. From the link, it sounds like the Fossil website is run entirely on such binaries (not surprising, considering the author).

I can confirm it does run on a single binary. I used Fossil for weeks, and its internals are just plain beautiful -- so much actually it almost makes me write websites in C++. (I know Fossil is in C.)

> That's why FastCGI, SCGI, WSGI, and other similar protocols all became popular at the same time as Rails and Django.

FastCGI and SCGI were popular long before Rails and Django, they predate those by at least 10 years and were popular long before those appeared.

WSGI is not even remotely relevant to the discussion, from your mention of it I can only surmise you don't have any idea what you're talking about.

Executing a C program with some non-trivial shared-object dependencies may also incur some significant overhead (dynamic linkage). But it is of course possible to statically link.

There's the really weird aversion to fork among people who don't understand just how fast and efficient fork is on Linux (and BSD) systems. It's an extremely common misconception, that occasionally leads to people over-complicating their solutions to problems.

I see it with databases, too...people often want to interject MySQL or another heavyweight database into situations where a flat text file would be vastly faster and vastly more efficient.

It's a problem of thinking that solving problems in the large is the same as solving small problems.

Fork goes out the window if you're connecting to a database, need to keep any structures in memory, or can envision a need to work on Windows. This covers a heck of a lot of situations.

And fork is still kind of expensive. Say a CGI just prints out a few K; the forking time will be relatively substantial. If you can afford it, who cares, but it is an issue.

It also does not apply in a heck of a lot of situations.

High-traffic websites ? He said quarter-million requests a day, so 3 requests per second (and inetd forks a process per connection, not per HTTP request).

Well, high-traffic by certain standards. Higher-traffic than what seems to cause many server setups to fall over, anyway; front-page on HN or Slashdot doesn't send more than that much traffic, and yet a bunch of sites seem to be unable to handle it.

To his credit, he sort of clarified that by implying it's "high traffic - for the sort of website that sits on 1 of 20 VM on a single physical server". If you do the naive multiply-everything-out-assuming-it-all-scales-linearly, 3% cpu on 1/20th of the physical server suggests an upper limit somewhere in the region of 165 million requests per day - that'd certainly count as a high-traffic site...

Reply on news.ycombinator.com