Comment by pcwalton | Hacker News Reader

Comment by pcwalton | Hacker News Reader

Parent Full thread

Signal handlers are pretty much impossible to write directly in a managed language (at least without heavy contortions from the runtime), because you could get a signal during a non-GC-safe point and then deadlock or corrupt your program. For example, malloc often takes locks internally; if you get a signal during malloc with that lock held and then try to malloc in your signal handler you can deadlock. Managed languages like Go treat allocation as an implementation detail rather than something the programmer has to explicitly invoke, making them basically incompatible with signal handlers.

(BTW, this is mostly a problem with signal handlers as a concept, not with Go.)

> The problem with using the syscall module is that Go does not export Fork and Exec. It exports ForkExec, because it needs to ensure they run on the same thread.

And because you can't safely allocate after a fork in managed languages, because fork drops your other threads, leading to the same problems above. :)

So the way Python does it is that it doesn't call back into the interpreter loop from the signal handler. That would be a really bad idea as you note, because the interpreter can call malloc and do all sorts of other stuff.

It queues your Python signal handling function, returns from the signal handler, and then runs it on the main loop at the next tick.

As far as I know this works fine. I wrote the same program in Python and it seems to work just fine, and have more functionality than the Go version. I guess one problem is that if you do a long blocking syscall on a tick in the main loop, your signal handler can be delayed indefinitely. There might be some hacks around this in the Python interpreter, but I don't recall offhand.

Yeah, that's basically having the runtime advance to the next GC safe point before running the handler. It's not direct control over signal handlers, but it's fairly low overhead. I guess Go could do this too if it wanted to. I don't think it works for fork though.

Yeah the problem with Go and "async processes" doesn't have to do with GC -- it has to do with M:N threading. I'm glad Rust abandoned M:N threading.

Though it is definitely a killer feature of Go to write in a blocking style rather than in a state machine style... but I feel that there might have been a way to do it without baking it so hard into the runtime (?). Maybe like coroutines in Python, which run on a single thread and are optional.

M:N threading causes a lot of other complications too, like calling from C to Go, etc.

A signal interrupts a blocking syscall. You can call PyErr_CheckSignals() that may execute Python signal handler before you restart the syscal on EINTR in C code. Look at how os.waitpid() is implemented: https://github.com/python/cpython/blob/master/Modules/posixm...

Long blocking syscalls will return with EINTR when a signal is received, at which point the interpreter should be checking the signal flag. The real problem with this scheme is if you call into some long-running C extension code, in which case the interpreter doesn't get a chance to check the signal flag.

Yeah doing something else on the main interpreter loop, BEFORE checking the signal flag, is what what I meant. The "something else" could be long-running C extension code, or a blocking syscall. (I realize the original signal will interrupt a syscall, but there could be another one after that.)

In either case, the signal handler doesn't get run for an arbitrary period of time. And another signal could occur.

I didn't trace through the code to see how this is handled... but it seems like it is hard to handle simply and correctly (you're kind of recapitulating the problems of signals to begin with, in a higher level language).

Any docs / clarity on this would be interesting to me, as I will probably implement this in my own language soon.

This is really tricky to get right: it's kind of like the problems you get with threads, except with the added bonus that you can't even wait on a lock, because the lock might be held by the thread that got interrupted. The easiest thing to do is just ignore the problem and not let users install signal handlers written in your language. The next easiest thing is the Python model, and it's pretty much the only option if your runtime is not multithreaded (as Python's is not: thanks to the GIL only one thread can be running Python at a time). If your language runtime is multithreaded, you can try to set things up so all signals are routed to a special signal handling thread that spins in a loop waiting for a signal handler to set a flag, then runs the appropriate handler.

Writing any code in a signal handler besides setting a flag is a very bad idea, specially when writing code across UNIX compliant OSes.

Unless it changed, POSIX gives no guarantees of stack size and function calls re-entrancy guarantees depend a lot on the OS and compiler versions.

Hence the best approach in managed system languages is just to mark a signal as active and re-trigger it outside the handler when control gets back to user space.

Reply on news.ycombinator.com