Hacker News Reader

Hacker News

Comment by 2bluesc | Hacker News Reader

Full thread

2bluesc

For those running `make`, add this to you shell init to set the concurrency level via env var:

> expert MAKEFLAGS=-j$(nproc)

View on HN

Good idea.

make -j without a number should do that, instead of spawning processes like there’s no tomorrow.

To be fair, having some way to spawn processes like there's no tomorrow is a good way to make sure there are no concurrency issues in your makefile.

I had a makefile with a race condition that triggered only sporadically. However, giving it more threads significantly increased its rate of occurrence.

Any doc that suggest this behaviour?

From the man page: > If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.

What this practically means is that a plain "make -j" will spawn all the processes that could run concurrently. So if you're building 1,000 files, it'll spawn 1,000 processes. This can either be fine, or completely terrible, depending on the number of CPUs you have an the amount of memory each process uses.

*export

As a software developer one might also choose to use the switch -O as to not mangle the output in a (too) nondeterministic order.

Or prefer ninja overall.

Some software doesn't support that - building that way can randomly fail with very confusing errors.

Then that software should add the proper dependencies to their makefile.

  make -j$(($(nproc)+2))

Why 2? I thought one process was forking off the other processes, not two.

You want to have more processes running than cores, so that if one process is waiting for i/o another can use the CPU time for something.

Reply on news.ycombinator.com