Comment by mrshoe | Hacker News Reader

Comment by mrshoe | Hacker News Reader

There's a good reason most Ruby and Python projects don't rely heavily on system calls: portability. If you care at all about portability, it's just easier to not hit the system calls directly, otherwise you'll have to detect host OS and make sure you're using the system calls properly.

If threads are "out", then pre-fork is way out. Just look at the history of the Apache project. I realize this all happened before the RoR era, but Apache used to use a pre-fork MPM almost exclusively. In more recent years it has added the threaded MPM and the async MPM. They have also put in the work I mentioned above to achieve cross-platform compatibility.

I'm just using Apache as an example here; I'm not suggesting that we should all use Apache. It's just funny to me to see a post that basically says, "All the stuff we've been doing for the last 5 years is out. We should be doing the same stuff they were doing 15 years ago, but in Ruby this time around instead of C."

Maybe software trends are like music trends? Everything from 5 years ago is lame, but the stuff from 20 years ago is super groovy, man.

Counter example: nginx, which uses fork(), and seems to smoke Apache while offering features like binary reloading without dropping connections (not sure if Apache supports this but I seem to remember no - please correct me if I'm wrong).

Portability is not always necessary - when talking about fork() and friends you're talking about trade offs.

When I'm only ever deploying to Unix environments, I accept the lack of portability in exchange for features I value.

That's not really a counter example. nginx does indeed fork off child processes, but it uses async I/O in each child process. It doesn't use the traditional pre-fork process-per-connection model that Apache's pre-fork MPM uses or that Unicorn is using.

nginx relies heavily on system calls at the expense of portability.

Oh, you were referring to the syscalls/portability tradeoff. In that case it's still not really a counter example, because Apache made the same decision in its early versions (Unix only). And that's a reasonable decision to make given the tradeoffs, I think.

The point that I was more upset with was the "threads are dead, pre-fork is the way to go" section of the article.

However, thinking about it more now, even the syscalls vs. portability arguments presented here are another example of forgetting the past (or just never being aware of it). After all the work from the Python devs to encapsulate syscalls in the standard library and provide a portable API to them, Jacob Kaplan-Moss says, "I’m a bit dismayed to see [syscalls] relegated to the dusty corners of our shiny dynamic languages."

Hello!? nginx is a POSIX network application written in C. Of course it uses syscalls. As the previous poster pointed out you are also wrong in that nginx uses anything like a prefork model. It _may_ fork of a process or more if it detect that it is running on a SMP system to take advantage more than one CPU. But each process has a strictly async io architecture handling request/responses within a big loop.

The point of the post you are responding to is that it uses syscalls. I'm not entirely certain what about that warrants a "!?"; could you enlighten me?

The '!?' was aimed at Mr. defunkt who said that

   nginx relies heavily on system calls at the expense of portability

Its a oxymoron as you cannot expect a C application that mostly do network and disk i/o not to use syscalls. That said, the author of nginx does an admirable job of reducing the numbers of syscalls and make the application as efficient as possible. In this context a syscall i.e. a kernel call is heavy and something one want to minimize.

The blog post on the other hand was talking about using syscalls from Ruby. I can understand Ruby programmers who wrinkle their nose at this. If you want your Ruby application to be portable then using, for instance, fork is not the best idea. In fact, programing anything long lived such as a server in Ruby is not the best idea. The GC in 1.8.x leaks memory over time and it is a common case that Ruby servers has to be restarted often as they consume all memory on the machine they run on. Adding fork on top of this is bad. A fork will copy the whole Ruby interpreter into the new process and you will end up with a lot of top-heavy processes - not a good idea unless you sell RAM. Basically, using Ruby as a systems programming language and for applications that are long lived is a bad idea.

Nonsense. My multitudes of small web services, many of which have been running months without a restart and while taking up no more memory than they did at the start say you're wrong. Even if there are some obscure bugs, and there undoubtedly are, that doesn't make ruby an unsuitable language for systems programming.

My only reply to this: Ram is free. You are optimizing the wrong thing.

> nginx relies heavily on system calls at the expense of portability.

Nonsense; Unix is quite portable.

Do you have any idea what a syscall is!?

Making nginx portable means using more syscalls -- the ones specific to the kernel you're calling. Across Unixes that just means using the appropriate epoll/kqueue/etc, for Windows support it means a total refactoring to use NT's Completion Ports.

When the author says that threads are out, he means that they are a difficult way to write concurrent code. Pre-fork is just one (classic) example of how to write concurrent code without shared state. Instead of threads communicating via shared state, you have processes communicating over pipes or sockets.

That said, you're right about portability. The attraction of Ruby's Thread is that it works identically across operating systems.

Reply on news.ycombinator.com