Hacker News Reader

How can Ant be bad? It's just Make in XML ... ;-)

But because it is Make written in Java with a XML syntax, it has inherently the same problems.

Fun fact: They had for years a makefile rant on their home page which boilt down to "Ant was invented because its developer couldn't make Makefiles work as his editor didn't properly show tabs vs spaces": https://web.archive.org/web/20100203102803/http://ant.apache...

Ant is actually a different beast, it is more declarative than procedural Makefile language.

Still ugly and terrible and limiting.

How is it more declarative than the Makefile language?

You declare targets and dependencies and the steps within are target a executed procedurally in sequence.

This sentence is 100% true for both Ant and Make.

Ugly, yes. It was developed during the times when everything had to be XML. But also Makefiles don't win a beauty contest.

Terrible. Maybe, but not worse than trying to write portable Makefile for something non-trivial (e.g. libraries).

Limiting? In what way? You are encouraged to use the supplied tasks which cover a lot. Creating new tasks would be done in the language you programming. You can still execute any arbitrary command if you really want to.

In ant, to declare a target for any no direct code generator is a royal pain. Even worse than in Make.

Ant is declarative in that you do not directly refer to files except in tasks that directly handle files. You cannot say that phony task x depends on file y.

I always suspected that ant was written by people who didn't first take a look at what already existed. Make's use of tabs was a mistake, but not a fatal one.

I actually like tabs, but if they bother you, you can use semicolons instead.

I do like the straightforwardness of Make. But it doesn't do quite enough for it to be simple to use for C or C++ programs. The simplest generic Makefile I can come up with that handles header file dependencies correctly is 10-20 lines of relatively complex interaction with the compiler.

But then again maybe that's just an argument against C and C++'s compilation model. :)

To be fair, dependency tracking is properly a function of the compiler, you can't do it in make alone without the ability to parse the language. And it's not 10-20 lines. GCC and clang both support the -M family of arguments that do this for you. It's a matter of running it to generate some sort of dependency file and including it in the makefile.

Yes, that's annoying and definitely a problem with the inclusion-based dependency model of C. But really it's not such a big deal.

Right, I was referring to adding the right -M arguments, including the generated .d files, and handling the edge cases with adding/removing files to the build. Overall the minimal Makefile I copy-paste into every new C project is about that size, so I suppose 10-20 lines includes a few other things as well.

> It's a matter of running it to generate some sort of dependency file and including it in the makefile.

This is not as simple as it sounds. Make has a fundamental limitation that dependencies cannot be generated on the fly, and current workarounds for dependency-generating dependencies are very messy [1].

[1] http://make.mad-scientist.net/papers/advanced-auto-dependenc...

>Although it’s simple, there are serious problems with this method. First and foremost is that dependencies are only rebuilt when the user explicitly requests it; if the user doesn’t run make depend regularly they could become badly out-of-date and make will not properly rebuild targets. Thus, we cannot say this is seamless and accurate.

Not so fundamental, if you quickly make depend after changing includes. Why is that messy? I never got that "buy magic automation, sell simplicity" trading. Deps changed, deps have to be updated. Once updated, they just work.

Those arguments are not enough. For dependencies to be fully reliable, the compiler needs to emit information about where headers were not found, in addition to where they were found. Otherwise headers introduced earlier into search paths after a build do not properly trigger a rebuild.

* http://jdebp.eu./FGA/introduction-to-redo.html#CompilerDefic...

* http://cr.yp.to/redo/honest-nonfile.html

Honestly, that's just intractable. That can't be solved for the same reason that changes to the makefile itself can't be tracked. Changes to the structure of the build system aren't "dependencies" as commonly understood.

Almost all other systems have this flaw in some way as well. You can fool everything, because a system capable of not being fooled would literally have to reparse and rebuild everything from scratch.

Almost all other systems have this flaw because they require dependencies before a build. If recording dependencies after the build, the entire problem becomes very simple. See here:

http://news.dieweltistgarnichtso.net/posts/redo-gcc-automati...

The example above was of a change to the build structure (include path, specifically) which changes the dependency tree in a way that doesn't involve changes to the files themselves and will be invisible to make or other tools that use file timestamps. I pointed out that this was just one of a whole class of intractable problems with dependency tracking that we all choose to ignore. It can't be fixed, even in principle. The only truly sure way to know you're building correctly is to build from scratch. Everything else is just a heuristic.

Why do you think the problem can not be fixed?

Also, have you looked at the redo tool redo-ifcreate? http://news.dieweltistgarnichtso.net/bin/redo-sh.html

I would like to boast that I have clearly solved an intractable problem, since (as I said) I have tools that happily track this very thing. But the truth is that the idea that it is intractable is rubbish, rather than that I can achieve impossible things before breakfast. (-:

You've switched tack between "not such a big deal" and "can't be solved" in the space of two messages. Both of your extremes are wrong. The truth is that this is simply difficult and needs either improvements to compilers or, as in my case, add-on tools that replicate the compiler's pre-processing and emit both the redo-ifchange and the redo-ifcreate information.

The following sentence about my redo implementation is wrong:

> In 2014, Nils Dagsson Moskopp re-implemented Pennarun redo, retargetting it at the Bourne Again shell and BusyBox.

I targeted the Bourne Shell (sh), not the Bourne Again Shell (bash). Also, my redo implementation contains redo-dot that paints a dependency-tree – I have not seen this otherwere.

You only put /bin/sh as the interpreter. You still used quite a number of Bourne Again shell constructs in the scripts themselves. Some are not in POSIX sh and utilities, some are not in the Bourne shell. (The Bourne shell is not conformant with POSIX sh, so targetting POSIX sh is not the same as targetting the Bourne shell.)

These include amongst others local, $(), >&-, the -a and -o operators to test, and command. (You also failed to guard against variables expanding to operators in the test command, but that is a common general error rather than a Bashism.)

Beware of testing against /bin/sh, even the BusyBox one, and assuming that that means POSIX compliance, let alone Bourne shell compatibility. Even the OpenBSD Korn shell running as /bin/sh or the Debian Almquist shell running as /bin/sh will silently admit some non-POSIX constructs.

While you are right about POSIX problems (like using “local”) I actually targeted Dash and older versions of BusyBox – not Bash.

I plan to work on POSIX compatibility for my redo implementation.

Make should have included something like tcl, or shipped its own tcl and its own os/posix libs. It is very easy for makefiles to become tied to a particular OS/Distribution/dev environment.

GN Make 4.0 actually has Guile Scheme. This is pretty recent and I haven't seen it used yet. If anyone has seen it used, I'd be interested.

https://www.gnu.org/software/make/manual/html_node/Guile-Int...

https://jaxenter.com/gnu-make-4-0-adds-guile-output-sync-bre...

FWIW I wrote about how Make, shell, and awk heavily overlap in functionality here:

http://www.oilshell.org/blog/2016/11/14.html

This probably won't happen for quite a long time, but I'd like to expand my Oil shell to have the functionality of Make.

On the one hand, it's kind of ironic that you're asking for Tcl, when shell is Turing complete and already such an integral part of Make (every line of the Makefile spawns /bin/sh).

On the other hand, I understand that shell is a language with many sharp edges and most people dislike it. The point of Oil is to get rid of the sharp edges, and then maybe shell can take the place of Tcl/Guile.

It seems a little ridiculous to write build scripts in make, shell, and guile, all of which are Turing-complete (not to mention Awk or Perl, which often show up). And I don't actually think Lisp/Scheme are very good languages for Unix-like text processing and syscalls/OS integration.

What happens when /bin/sh is dash or bash? I despise /bin/sh, it is basically PHP. There isn't a terrible amount of basic functionality that would enable makefiles to cross platform (even cross distribution would be nice). Interacting with files, file metadata, processes, exit codes, etc.

Absolutely love your oilshell posts. Keep it up. I think a concatenative language would be great as the embedded logic for Make, but a Lisp would also work.

Yes that is one of the problems with Make -- /bin/sh is different on different machines, and that information appears nowhere in the Makefile. There's no way to test that the commands you're running will actually work on someone else's machine, other than by "memorizing the manual".

Combining make and shell would mitigate this problem. Then you would need one binary in sync instead of two. Well, I suppose you also need "busybox", e.g. cp, mv, mkdir -p, etc. But in practice I think that is less of an issue than shell and make colliding.

Thanks for the encouragement! (If you didn't catch it I linked these build system observations in a sibling comment: http://www.oilshell.org/blog/2017/05/31.html)

I've played with concatenative languages, and read a lot about Forth. I'm not sure I see them as great for "logic". They are very elegant for certain problems, but fall down for others (IIRC the quadratic formula was a popular example.)

The main kind of logic you need in Makefiles is expressing build variants -- e.g. for debug/release, internationalized builds, coverage builds, profile-directed feedback, running parameterized test suites, etc. I think that can be done fairly well with a Python-like imperative language with dicts and lists. (Some people have mentioned Bazel, and it is derived from Python and works fairly well for that.)

> /bin/sh is different on different machines

Write portable shell scripts (or lines, in the case of a Makefile).

There's some tips here: http://people.fas.harvard.edu/~lib113/reference/unix/portabl... and of course you can refer to http://pubs.opengroup.org/onlinepubs/009695399/utilities/con...

That's not useful advice. That's what I mean by "memorizing the manual" (or POSIX spec).

There's no way to test that the commands you're running will actually work on someone else's machine, other than by "memorizing the manual".

I have an entire book on this:

https://www.amazon.com/Beginning-Portable-Shell-Scripting-Pr...

It goes through all the commands and common versions of Unix and which flags are likely to work, etc.

Even that book admits there's a lot of folklore, because nobody has actually gone and tested things recently. Something as simple as safely writing with "echo" is a problem. You can argue that any script that uses echo $foo is incorrect (because $foo might be a flag). Conversely 'echo --' is supposed to print -- by POSIX.

The only people who are likely to even attempt this are people whose full-time job it is to write shell scripts, or the authors of tools like autoconf, which must generate portable shell. autoconf shell is a good example of the anachronisms and hoops you need to jump through to support this style.

Nobody else has time for that, because they have to spend their mental energy writing C and C++, not shell and make. So that's why we have pretty low standards for the quality of build systems. The tools aren't there to support writing a robust and easy-to-use build system.

Meta, I really wish I could reply to both chubot and stephenr ...

There is way too much incidental complexity in /bin/sh and almost all build systems. Unix is awesome, but I also hate Unix for being "good enough". Unix Hater's Handbook and all. I hate it from above, not below.

We should always be seeking to reduce the cognitive burden in the tasks we do. One level of fail for /bin/sh is the level of semantic density and the lack of discoverability. It violates the principle of least surprise like nothing else I have used. Look at variable assignment!

    export FOO2= "true"
    export FOO3="true"

These are semantically different. One evaluates the string, the other does not. And by being so obtuse, the majority of folks randomly mutate their sh scripts until they appear to work. No knowledge gained and nothing that would qualify as engineering.

The biggest problems with sh and Make are that lack of debugging and introspection. Any follow on tool that would displace them should make developer ergonomics the highest priority.

does the faded color mean this post is downvoted? In that case, why is it so?

It does, and I don't know why. I mostly ignore votes, and usually only vote when I think a viewpoint needs to get more exposure, not because I think it is "right".

SHELL=/bin/bash

> What happens when /bin/sh is dash or bash? Dash is specifically designed to be a posix compatible shell for use as /bin/sh, i.e. in shell scripts (or called by makefiles).

When Bash is invoked as "sh", it runs in posix compatibility mode.

There is a reason some of us keep harping on about "don't write bash-scripts, write portable shell scripts".

> I despise /bin/sh, it is basically PHP.

Similarly, I despise chickens. They're basically the sub-prime mortgage crisis.

The guile integration in gmake is a bit shallow, but still quite useful (here's the whole implementation [1][2]). It could also use more real-life usage examples - there's really nothing out there besides a few short snippets.

As a learning experience, I wanted to implement parts of the GMSL[3] on top of guile. It looked like it could be useful to others, so I made it into its own project: the GNU Make Toolkit[4] (it's still very much a work in progress - I've put more effort into the test suite and the documentation generator than into actual functionality).

[1]: http://git.savannah.gnu.org/cgit/make.git/tree/guile.c

[2]: http://git.savannah.gnu.org/cgit/make.git/tree/gmk-default.s...

[3]: http://gmsl.sourceforge.net/

[4]: https://github.com/gvalkov/gnu-make-toolkit

This is where cmake missed the boat, imo. They were even using Tcl/Tk for a medical imaging project[0] when they decided they needed their own build system. Then when they needed an expressive embedded language, wrote an ad hoc one from scratch.

[0] https://www.kitware.com/platforms/#itk

Schaudenfreude Lol! I don't like the term "too smart for one's own good" but sometimes it applies.

The CMake scripting language [0] .

[0] https://cmake.org/Wiki/CMake/Language_Syntax

POSIX even has a number of implicit rules that make should recognize ( http://pubs.opengroup.org/onlinepubs/009695399/utilities/mak... ).

The real problem I have is that make generally relies on shell commands like cp, so there's no truly cross-platform (e.g., Windows and POSIX) way to copy files around in a makefile. I guess you could build your own utility, use it, and delete it when you're finished building. I've actually resorted to using ExtUtils::Command ( http://perldoc.perl.org/ExtUtils/Command.html ).

make does have a platform independent strict mode. Unfortunately the ".POSIX:" mode is sufficiently limited that you won't want to use it for anything you're writing by hand.

Wow. I didn't realize this. Does it support Windows, without Linux etc installed?

Don't know if there's a Win32 build of GNU make, but Microsoft has nmake. I haven't written a Makefile for it in years, but I have a hard time imagining there being much portability between the two. Not in the Makefile stucture, but rather in the commands it invokes to actually build targets.

I'd guess that nmake is POSIX-compliant (or close to it), simply because the POSIX requirements are so minimal that it would take quite a bit of effort to avoid being POSIX compliant.

msys2 / cygnus, etc have GNU make for windows.

I sometimes think we developers collectively gravitate towards complexity just to feel smart.

Did you enjoy build.rs?