Hacker News Reader

Comment by caf | Hacker News Reader

In my opinion, it's libraries that secretly use threads under the covers (but still access state shared with the rest of the program like the malloc heap) that are what's dangerous.

View on HN

Yes. Libraries should provide algorithms and data structures, and parameterize a threading interface if necessary (though I think it's usually not necessary).

Applications can then instantiate their own threads and pass them to the library (i.e. dependency injection.)

So if you have 10 different concurrent libraries in an application, then you don't end up with 10 different threading paradigms or 10 different thread pools. This is a matter of application architecture -- libraries shouldn't bake in a specific policy.

true...

...this applies not just to threading, but other things that are not in the domain of the library such as memory allocation strategy or communication mechanism.

Libraries should be small, focused and decoupled. This is the way to proper re-usability.

Like left-pad?

jeez, no.... extremes of everything are terrible.

I suppose another rule should be that there libraries should encapsulate a sufficient amount of complexity to make their existence worthwhile.... unlike left-pad.

That's a function deployed as a library, not a real library; the word library implies many functions.

There's nothing really wrong with packaging small isolated pieces of code. The debacle came from consumers' bad habit of randomly upgrading their dependencies without doing full regression testing (partly because the tooling encourages this terrible practice).

I know it's very different in JVM land, but many Akka libraries do something similar where they take am actor system, allowing the developer to share or separate the actor systems.

> Threads are inherently the right way to do concurrency for more complex tasks.

True, but they're more often used as a crutch where a select-style loop and/or state machine would be more suitable, leaving race conditions in their wake.

Libraries should not be disallowed from spawning threads. Threads are inherently the right way to do concurrency for more complex tasks. fork() is a tremendously complex call and has huge implications on all things in the OS.

For many years now I have the rule that new code must not use fork() and I am living a happier life. It's a bad call and it has no use if you have good threading support in the language. There are many programming languages that do not support fork() andy more or heavily restrict the use.

I would add, in many/most cases using fork() without calling exec() just doesn't make sense. Even at the system call level you have clone(), which is how actual threading is implemented under the covers. And really, if you need a thread, just get a thread. I see extremely few or no advantages to doing fork() instead of spawning a new thread normally using whatever interface is provided (except in the case of exec() where fork() or vfork() is necessary).

What is dangerous is that the interactions between threads and processes aren't part of POSIX standard and its behavior is OS specific across UNIX implementations.

This isn't an issue on non-UNIX OSes.

How many non-unixes do have fork_without_exec()?

Most non-UNIX OSes only have exec(), not fork().

Hence why fork() is just kind-of implemented on their POSIX compatibility layers.

In UNIX it is only an issue, because IEEE doesn't want to set in stone in POSIX how threads, signals and processes should interact, due to the way threads were added to UNIX.

Non-unix systems don't have exec; they have spawn.

Windows has exec.

Exec is substitution of current process with new image without closing fds or changing pid. Which winapi function does that?

_wexec and friends. It internally uses CreateProcess obviously.

Right, but it's not used in the sense of fork+exec. I should have said that Windows doesn't only have exec, it has spawn too.

Yeah, sure. I always think of spawn as a plain wrapper exec/fork, hence why I wrote it like that, but you are right correcting me.

Pthreads are part of the POSIX standard. Hence the "p".

Then there is pipe() and probably some more similar ones.

Then you should read what POSIX says about pthreads, regarding mandatory and optional API support, specially the implementation semantics and UB.

For example, what happens to the interactions of signal handlers semantics and thread scheduling.

Or what happens to the threads when the process does a fork().

> Or what happens to the threads when the process does a fork().

You mean this? http://pubs.opengroup.org/onlinepubs/009695399/functions/for...

  A process shall be created with a single thread. If a multi-threaded process
  calls fork(), the new process shall contain a replica of the calling thread
  and its entire address space, possibly including the states of mutexes and
  other resources.

Granted, it's SUSv3.

I would advise people that lack experience writing portable UNIX code to read "Advanced Programming in the UNIX Environment" by W. Richard Stevens and Stephen A. Rago.

Just to get a glimpse of what POSIX says and what each UNIX actually does.

EDIT: Not talking about you, rather in general.

A thread belongs to a process so, for example, whatever the "second" thread does is the same as what a process does.

POSIX defines a few mechanisms for IPC.

What is optional and what is mandatory in pthreads doesn't concern me as i don't even like pthreads. (will still read what you said, later)

CSP stile concurrency over pipes (be they named or not), shared memory and synchronizations over it. POSIX defines mechanisms that work as expected over UNIX platforms.

I wonder how many (standards compliant) programs work differently between linux, *bsd, solaris, etc.

> A thread belongs to a process so, for example, whatever the "second" thread does is the same as what a process does.

Wrong, you can fork() and only the thread issuing the fork () call survives the fork(), for example.

There are quite a few other differences.

> POSIX defines mechanisms that work as expected over UNIX platforms.

Not really, there are a few semantic differences and corner cases specially outside the GNU/LINUX, *BSD ones.

>Wrong, you can fork() and only the thread issuing the fork () call survives the fork(), for example.

Thinking about it, it makes perfect sense.

Threads are processes that share the same memory space. When a thread is created (via pthread_whatever() or clone()) it starts executing some code that is practically in a $RANDOM place. So how would one go about copying the state of a process that has threads ? Freeze all the threads and copy everything ? That way the thread, that is probably doing some job at the moment, would end up being cloned without it knowing. So you would get two processes (threads are processes) that are doing exactly the same thing. Until something external changes their (internal) state, that is.

Non-determinism should always be avoided if possible.

This still does not say that one couldn't write portable, threaded, UNIX compliant code. Nor that those undefined behaviors from the specs would be fixable by rewriting the spec (by adding atomic fork+exec, that seems to me to be the context of this whole thing).

I dunno--under that definition every concurrent GC is "dangerous".

What libraries need to do GC? That is a language issue and not a library issue.

When you are embedding a language like Go in a C program, its runtime becomes a library. But that's the example the proves the rule: there are (or were) well-known problems with that. Libraries shouldn't be doing GC.

> What libraries need to do GC? That is a language issue and not a library issue.

Boehm GC disagrees.

I completely disagree. The only part of this that seems dangerous is the fact that it can turn a single-threaded program into a multi-threaded one, and the only real problem there is it makes fork-without-exec unsafe. But fork-without-exec is basically unsafe anyway unless you can absolutely guarantee that nothing has ever spawned a thread. Meanwhile, preventing libraries from using threading under the covers basically means throwing away lot of performance gains for no good reason.

This is the very first time I have heard about fork-without-exec as dangerous, but it is not the first time I have heard about having to be careful about libraries and hidden threads.

The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.

> The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.

That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread to do that without causing any observable difference in behavior (beyond potentially turning a single-threaded program into a multithreaded one). And if I'm on a platform like OS X or iOS, I can use higher-level threading APIs like libdispatch that make it even simpler.

This is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour.

And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to.

    #include <thread.h>
    #include <yourlib.h>
    #include "my_eventloop.h"

    void launch_da_ting()
    {
          std::thread ting_doer([]{
              Result rez = yourlib::do_da_ting();
              my_event_loop::post([rez](){get_rez(rez);});
          });
          ting_doer.detach();
    }

Now it's true those five lines would become a single function call if your library forced the user to user a particular dispatching library. But I think that would be poor seperation of concerns.

It's leaky in one extremely minor way that affects almost no programs whatsoever. It's extremely rare for anyone to do fork-without-exec, and anyone doing that already has a large number of restrictions they must be aware of.

And no, the user can't threadify that themselves. Your sample code is putting the library call on a thread. That's not at all what I was talking about. I was talking about a library using multiple threads (or, more likely, some abstraction over a thread pool that handles scheduling chunks of work on threads for you, such as libdispatch on Apple platforms) in order to process a bunch of data in parallel, data that only the library knows about. The user makes a single call into the library, the library processes a bunch of data in parallel, and then returns to the user. From the user's perspective it's a synchronous call, but it runs in a fraction of the time it would if the library wasn't allowed to use threading internally.

You can have your cake and eat it too -- there's no throwing away of performance gains. Libraries can use threads, but not directly or "silently". They should take threads as parameters. Then the application knows when they're being used and the application programmer can reason about it.

Yes, in theory. In practice, every library developer/maintainer uses different guidelines.

In practice, a lot of library developers/maintainers read this list, or know someone who does.

compilers and runtimes can spawn threads implicitly when required (think of openmp, cilk+, or even autopar). Are you saying that libraries shouldn't use these facilities?

If you're spawning threads implicitly then you're no longer a library, you're a framework. Sometimes a framework is a good fit, but frameworks don't compose (if you try to use two frameworks in the same application you're gonna have a bad time) so there is a cost. But in any case, better a framework that's explicitly a framework than a "library" which can't actually be used freely in general-purpose code.

would you consider a parallelized BLAS a framework?

Yes. Isn't that what I just said?

well, then you have a pretty non-standard definition of framework.

> If you're spawning threads implicitly then you're no longer a library, you're a framework.

That't not what makes a framework. It's all about inversion of control, you call a library and a framework calls you.

http://stackoverflow.com/questions/3057526/framework-vs-tool...

If it's spawning threads, it's controlling the control flow. Either it's calling you, or the control flow is out of control.

No. His definition is correct. If you make the call, it's a library, that it might use threads during that call does not make it a framework. Libraries don't mean you're in full control of everything, they only mean you invoke all interactions and get results back rather than you subclass something and plug a part into a framework. If you have to subclass something, it's a framework.

I've used OpenMP a little bit, and I would probably avoid it for libraries.

I might use it to start out an application, since it requires very little code for dumb parallelism. But if I were making a real library, I would try to abstract the concurrency policy away so the user can control it.

This should be trivial by layering the library. Just provide a bunch of single-threaded functions, and then some OpenMP wrappers. Then user can call the single-threaded functions if she wants to do something different.

What I don't want is OpenMP baked deeply into the program logic. I think it shares the problem I was talking about. If you have 10 different libraries using OpenMP in an app, and the app is ITSELF multi-threaded, it seems like you have a mess of threads that you have no control over.

guh. Those bugs aren't fun to hunt down. I don't think you really understand the importance of avoiding global or semi-global states until you have to do concurrent stuff, or when you have to start being serious about tests.

Reply on news.ycombinator.com