Hacker News Reader

Comment by madez | Hacker News Reader

> In particular, it is not safe to call exit, free, ioctl, putchar, or perror from a signal handler.

Why is it accepted by the compiler, then?

This is a good question and doesn't deserve to be downvoted.

I think the reason must be that the problem lies in the intersection of three areas of development (the C programming language, the C standard library, and the POSIX operating system definition) and so requires coordination to solve.

Think about how you would implement warnings for failures of async-signal-safety. One approach would go like this:

1. In compiler front-ends, introduce a new attribute that can be attached to function declarations to indicate that they are async-signal-safe, for example __attribute__((async_signal_safe)), and a new attribute that can be attached to function parameters and struct members to indicate that the passed value must be async-signal-safe, for example __attribute__((require_async_signal_safe)).

2. In C library headers, apply the async_signal_safe attribute to all the async-signal-safe function declarations, and apply the require_async_signal_safe attribute to the parameter to the signal function and to the sa_handler member of struct sigaction.

3. In compilers, propagate the async_signal_safe attribute, so that any function that only calls async_signal_safe functions is also marked with the attribute.

4. In compilers, check that when a function is passed to a parameter or assigned to a member with the require_async_signal_safe attribute, the function is marked with the async_signal_safe attribute, and issue a warning if not.

This doesn't solve the whole problem (sometimes the compiler will not be able to know which function is going to be passed as the parameter to signal, because this is determined at runtime), and it might have false positives in obscure situations (you might in theory use a struct sigaction for some other purpose and never pass it to sigaction) but it would catch many cases, including the one in beep.

But notice the amount of coordination required. It would require input from several people with different areas of expertise.

I'll be quicker to blame the signal API than the programming language on that front. Dealing with unix signals correctly and robustly is far from trivial and rife with footguns. For instance I believe that Rust still doesn't have a good general purpose solution for handling signals that doesn't involve the libc and unsafe code.

Signal is basically the crappiest form of IPC available on a modern operating system short of emulating a mouse and keyboard and typing into the other application's terminal window.

one of my favorite implementation of signal handling has always been DJB's qmail implementation (actually all his code incl djbdns, daemontools etc). His coding style has been a huge inspiration on writing easy to read and secure C.

> emulating a mouse and keyboard and typing into the other application's terminal window

There was a "talk"-like application for BBC Micro Econet called `*NOTIFY` which worked like that (minus the mouse, no mice in those days). You could message people, but also type commands as them. Happy days ...

The simplest way to write safe signal handlers is to only ever:

     - write(2) to STDERR_FILENO
     - write(2) to a "self-pipe" (i.e., a pipe where the same
       process is waiting on in its event loop), thus turning
       the async signal event into an async *I/O* event that
       can be handled without any constraints regarding
       async-signal-safety
     - _exit()

Yes, there are other async-signal-safe functions that can be called from a signal handler, but it's generally not worth it. Adhering to my more constrained approach will keep your code safe and will make it easier to always get it right.

ALSO, while we're at it, the only global or thread-local variables you can read from or write to from a signal handler must be of type volatile sig_atomic_t (or else volatile of any other integral or pointer type that you can use with atomic operations). This is very important. E.g., imagine using SIGUSR1/2 to manage verbosity levels...

Option 2 is very similar to how signal handlers work in Go. When a signal is received, a value is written to a channel and the library user is responsible for reading values from the channel and responding appropriately.

https://golang.org/pkg/os/signal/#example_Notify

It's the only sane thing to do.

A write(2) to STDERR_FILENO for verbosity/debugging is fine, but mostly you don't want to do this because it will interleave with any non-line-buffered stdio writes to it... An _exit(2) is also OK if you really want to do that, but generally you want to do some cleanup, so might as well do the self-pipe thing every time.

The only tricky thing is when you use SA_SIGINFO and you want to pass the siginfo_t data to the event loop. You can write(2) that to the self-pipe, but you have to be careful of the possibility that it will fill up. You can always create a new pipe(2), write(2) the siginfo_t to it, close(2) the write end, and send the read side fd via a socketpair(2) that the event loop listens to.

It is not the only sane thing to do.

kevent() is another way to handle signals. It puts handling them into the program's main event loop, which is done synchronously with normal event-dispatching mechanisms and so does not have worries about asynchronous signal safety, because with kevent() they are just another type of filter.

Sure, if you're going to use non-portable constructs, there're better alternatives to the self-pipe thing. Linux has something roughly similar with signalfd.

The nice thing about the write-a-byte-to-the-pipe thing is that it works virtually everywhere.

> ALSO, while we're at it, the only global or thread-local variables you can read from or write to from a signal handler must be of type volatile sig_atomic_t (or else volatile of any other integral or pointer type that you can use with atomic operations). This is very important.

I'd rather recommend C11 atomics without volatile. They use memory barriers to guarantee visibility, and getting the idea that "volatile" should be used in the context of atomics is a bad idea, as it allows reordering (unlike memory barriers).

Because the compiler doesn't and cannot know the semantics of signal(2)/sigaction(2). There's what POSIX says, which a compiler could enforce using heuristics, and there's what your C library says. It's perfectly plausible to make a number of nominally not-async-signal-safe functions from the standard actually async-signal-safe in a particular C library. Granted, if you were to take advantage of such an extension, your code would not be portable.

(EDIT: The compiler doesn't even know that you're using POSIX. It can figure it out contextually. It doesn't know which C library you'll be linking with though. Bottom line: we need some C standards extensions, or failing that, GCC/Clang C extensions in order to best handle this, though there are some heuristics a compiler could implement even without those, at some mild risk.)

(E.g., suppose there was a thread-local counter of signal handlers on the stack... then stdio functions might be able to handle async-signal reentrance. It's not too farfetched, though it's obviously a lot easier to not bother at all and just leave the set of async-signal-safe functions being just the set of system calls that have sufficiently thin stubs in the C library.)

Now, the C standard could have additional keywords, much like, say, 'volatile' and friends, to describe async-signal-safety, reentrance, and other characteristics of functions. And if the C library and your programs used these then the compiler absolutely could warn or refuse to compile your program when you break the standard's rules.

Incidentally, Unix/POSIX signals are horrible. The only sane and portable way to handle them in programs that do I/O is to have an event loop and a "self-pipe" that the signal handler can write into so that the event loop can pick up and handle the signal as a normal async event in a context where there are no constraints on calling async-signal-unsafe functions. This is what I always do in my programs. I strongly recommend it. This means you don't need ppoll(2), pselect(2), signalfd(2), epoll_pwait(2), and so on -- you don't because you're always turning signals into normal I/O events, so you don't need to worry about signal blocking, and you don't need to use less widely available functions.

It can't be done in general. A signal handler is just a function that at runtime is connected to a signal. With some work many or most unsafe handlers in the wild can be detected, but so can many other problems, I guess.

I'll bite. A helpful compiler would at an appropriate warning level tell you when you are setting a signal handler with a function you haven't (helpfully) annotated with some construct to say "this thing is a signal handler". Said helpful compiler would then also use this helpful annotation to forbid or loudly warn against a helpful subset of naught functions it is able to deduce are being called therein because human programmers cannot be reasonably expected to consume all 64 pages of POSIX function specs before calling any one of them. Seriously.

That's what I was saying. So why don't you step ahead and do it? I'm sure it will be not more than two weeks worth of work. And it will fatten up the compiler only a little bit. And it will annoy proficient programmers only a little bit. How would you deal with calling a function that is externally defined, so you can't easily check if it does something unsafe (or calls another function that does). I'd figure any attempt to check if a signal handler is safe would have to be either extremely expensive or pretty half-arsed.

(That's what I was saying, except the bit that programmers cannot reasonably be expected to understand what to do in a signal handler. That you should usually only set a (volatile) flag and return immediately, and should take extreme care if you really need to do more, is basically the first thing you learn when you read up on signal handlers. That's also completely obvious if you take a look at how they are implemented.)

Oh, I think we agree. Implementing the feature sounds like a right creeping pain for very little payoff. I would differ wrt "basically the first thing you learn when you read up on signal handlers" -- these days, the crazy kids writing code just start using the language and libraries. Learning why is usually the last thing learnt, and only when something, like a beeping-beep service, sets fire to something.

Yep, if programmers can't be bothered to read a few lines of manpage before using a new function as distinctly special as signal()/sigaction(), then I think there is no help in sight.

But on the other hand, the signals API might be a problem. Signals aren't exactly praised as a marvel of engineering. There probably is a need for something as low-level as that in some cases. But in the common case an API that just asynchronously sets a volatile flag when a signal is received, should be sufficient. Calls to "slow" devices are interrupted and return EINTR anyway.

Seems like a good job for static analysis/linting vs compiler.

Perhaps a better question is why signal handlers have these restrictions, and why they don't behave more like e.g. threads.

Other mechanisms have been designed, but they have different qualities. Probably none of them are as performant as signals, which might have been more important then than it is today. Creating a new thread for each single signal definitely feels wrong to me. Linux has signalfd() with which you can setup a signal-handling thread, but I think there are problems with that API as well. I think there is some value in being able to run a handler in the context of an existing thread the way traditional signals allow. (While I do vaguely remember that with common signals implementations it's very hard, or impossible, to control which thread is handling a signal).

Because they were modeled after interrupts, and can't be changed for historical reasons. (E.g. any code which relies on the atomicity of a signal handler with respect to its context thread would break.)

Signal handlers were always meant to be "top-half" interrupt handlers -- set a flag or put something in a queue for the main thread to process. signalfd(2) formalized this usage.

Why wouldn't it be? Why would a compiler know what a signal handler is?

Because signals and signal handlers are part of the language spec.

I was going to disagree and say that they are part of POSIX, not part of C99; but they are indeed part of C99, section 7.14.1 specifically.

Yeah, it’s a little surprising which parts are where sometimes, and if you spent most of your time on a POSIX system you can easily end up with the wrong assumptions.

This is a really good point actually, aren't there static analysis tools that can figure out things like this? I know in JS we have linters to stop you from doing things that are most likely not very smart.

I find this very useful, a lot more useful than to say "If you don't even read the manpages there is no help in sight".

Maybe the compiler should not do it but perhaps you could run a linter against packages that are shipped with an os to find issues like this easier.

It's pretty likely that there are tools which can (attempt to) detect this. The question is why aren't these tools used. The answer is because they are expensive, and using them implies development costs as well - learning the tool, setting it up to filter out false positives, setting it up to run in an automated way, structuring the code differently so that the tool is happy (which can lead to worse structure), making sure all the remaining false positives aren't problems every time the tool is run. It's not what you do in an open source project that has a few lines of C code.

C has separate compilation: the compiler isn’t guaranteed to have all of the relevant source code at one time.

When the signal handler is compiled, there’s nothing to say “this will be used as a signal handler,” so while the compiler has the source it doesn’t have a reason to complain about calling any particular functions.

Then, when that function is used as a signal handler, the compiler has the source to the calling code (well, the code setting up the callback) but may not have the source for the handler itself. So now it knows which rules apply but may not have the code it needs to enforce those rules. If they’re in the same file, it could. And while that is a common case, it’s not the only possibility.

How much should the compiler or linter know about the platform’s API? How can I tell the compiler about any arbitrary rule my own code must follow?

As someone else suggested, you could do it with annotations, but that’s nonstandard.

I don't think there's a purely technical reason a non-standard linter cannot exist, but it's probably a large investment to build such a tool (filtering false positives, etc.) and no one has made one sufficiently good and popular enough to be used for a simple one-file-utility.

How would the compiler know that a function was a signal handler?

It wouldn't. But it might be possible to make it impossible to call forbidden_function() from anywhere where it shouldn't be possible.

For example: the signal handler would be a function that accepts an argument of some kind, and the argument is the key to reaching all apis allowed from that point (e.g. the argument must be passed on to allowed apis, or the argument contains function pointers to all allowed functions, that is "OO-style").

I'm not saying this would be possible by any stretch of the imagination for POSIX.

How would you differentiate a function that takes an argument of some kind and handles signals from one that just takes an argument of that kind (e.g. for debugging, logging, or some other sort of introspection)?

Edit: to be clear -- I'm not asking this just to be snarky :-). Inferring functional information from nothing but semantic information with no functional implication is a very complicated problem that has very error-prone solutions. In C's case, in fact, I think C99 defines signal handlers as taking a single argument, an int. A compiler that doesn't let me call free from a function that takes a single integer argument wouldn't be too useful.

IRL, there are analysis tools that can help catch (a subset of) this sort of problem, but they are environment-specific and don't rely on just the function definition. They either look at the signal handler installation calls (but then they're restricted to information that's known at compile time!), or require some sort of annotation (e.g. via special comments a la Doxygen, or via macros etc.).

And even then, the sort of problems that they catch are specific to each environment. The restriction isn't that you shouldn't call <these functions> under POSIX, the restriction is that you shouldn't call functions that are not async-signal-safe (i.e. they're not re-entrant or they're not atomic with respect to signals). There are a few POSIX functions that are safe to call from signals (see man 7 signal-safety on a Linux box), but that list obviously doesn't cover user-defined functions -- some of which may be OK to call, others not so much.

You can probably determine if a function is async-call-safe with some static analysis but at this point, it's the sort of stuff that really doesn't belong in a compiler anymore...

To be clear: I wasn't suggesting that making everything context aware would be possible in either C or Posix. It would most likely require a complete rewrite of all APIs to begin with - including the C standard library. It would no longer resemble posix after that.

It's not something that can be solved on an API level. You can rewrite the C standard library all you want, you still won't be able to differentiate between this:

    void handle_signal(struct signal_handler_arg *arg)

which does signal handling

and this:

    void check_signal_info(struct signal_handler_arg *arg)

which validates a struct signal_handler, is called outside a signal handler, and doesn't do any signal handling.

Similarly, you would not be able to verify that this is okay:

    void handle_signalstruct signal_handler_arg *arg)
    {
        ...
        some_global_state->some_cb(...);
    }

If some_cb is populated at runtime, how do you know (at compile-time) that it's been populated with a function that's safe to call inside a signal?

> you still won't be able to differentiate between this > which validates a struct signal_handler, is called outside a signal handler, and doesn't do any signal handling.

Let's use a pseudo OO syntax:

    void foo(arg: argtype)
    {
        arg.do_thing(); // ok 
        frob(arg); // not possible - if frob might be forbidden anywhere then it's not callable like this
    }

That frob is forbidden isn't because the compiler detected that foo is a signal handler, but because no forbidden functions exist. I'm not sure this would be practical in this context - but it's the only way I can imagine to have a compiler detect that you don't call a particular function from a particular context. Basically, very few global/static functoins would have to exist, and no/very little global state as well.

I really don't know what you mean by this:

> That frob is forbidden isn't because the compiler detected that foo is a signal handler, but because no forbidden functions exist.

or this:

> not possible - if frob might be forbidden anywhere then it's not callable like this

I was trying to describe how an api would look where the compiler would make sure you could not call a certain function from a certain context. So I was trying to argue that then the api would have to be such that you can't actually import that function to that context.

So basically: say that putchar is the forbidden function. Then there exists no global function "putchar". There is no way to just "import" a header and call putchar in our language/api.

Instead, everything available to a function is what's passed to it (an interface to available API). So for example for a particular type of context (a handler, say) - a very restricted api is passed in as argument, meaning the code inside that function has very little api surface area to play with.

> I was trying to describe how an api would look where the compiler would make sure you could not call a certain function from a certain context.

It's not a certain set of functions that you want to forbid, it's a certain behaviour (e.g. being interruptible by signals, or being non-reentrancy). Forbidding functions won't get you anywhere. You can always forbid write(), but you won't be able to forbid functions that dynamically dispatch to write() at runtime, or user-defined functions that aren't reentrant.

Edit: i.e. when we say you shouldn't call write(2) from a signal handler, that's not because there's a list somewhere in a POSIX standard with functions that you shouldn't call in a signal handler and write(2) is on it \. That's because the POSIX spec allows write(2) to be interrupted by a signal and that's not okay inside a signal handler.

___

\ There might be one, I haven't looked at a POSIX spec in a long time now -- but either way, the point is that such a list is open :-).

I see. Preventing access to classes of behaviour is a harder problem. Reentrancy (single threaded - such as mutual recursion) must be really hard to detect with certainty other than at runtime, for example. In a multi threaded context it's obviously even harder.

Classifying functions into more categories like "interruptible", "uninterruptible" seems doable with enough language support. It doesn't look to far from what e.g. Rust and C# does with "unsafe", i.e. a status of functions that bubbles up so that your function may be tainted as unsafe by calling unsafe functions. If the number of classifications of functions is small as is the case with unsafe/safe, or uninterruptible/interruptible then it would seem possible to sort this out via compiler support and keywords for example. So for example a signal handler is then an uninterruptible function. It's a compiler error to call even indirectly, or dynamically dispatch to interruptible functions.

For any type of safety, dynamic dispatch obviously needs to be done in a type safe manner, so no pointer arithmetic/function pointer invocation can be allowed. Doing so would make the called code be assumed to be the worst of all categories: unsafe, interruptible etc). This is how unsafe/safe works too.

So while it's cumbersome I'm sure it would be possible to achieve at least to some extent, given an expressive enough language and a clever enough compiler.

You can teach the compiler. "How" is easy. "Should" is debatable. One can easily write a clang diagnostics pass to recognize few set functions.

A signal handler is not special, it's just another function. So the compiler can't really apply the rules. Also this is a very limited subsection of all the many things you are not supposed to do in a signal handler.

It’s undefined behavior, which means the compiler is free to accept it and do something sensible, or failing subtle ways, or make demons fly out of your nose.

Reply on news.ycombinator.com