Insane complexity in build systems just makes your life sad. Trust me, I know -- I write java for a living. We have ivy, maven, ant, gradle, sbt, leiningen, and I'm sure a few more.
The build complexity is one of the reasons I stay away from Java: If these people think they need XML for builds, what other needlessly complex horrors have they perpetrated? And that sort of thing.
Do you have your buildscript template on github?
Limited build tools are a good thing.
All you have to do now is make sure your hands are the right hands.
Like buddha said: right mind.
Let's say we have a make format called dmake. It invokes $CC with the specified arguments for each file, and links them together into a binary/so/whatever, putting it into the build directory and cleaning artifacts. Okay.
Now say that you start a new project in rust. Well, crap, dmake doesn't work. You have to use rdmake, which is built by different people, and uses a more elegant syntax - which you don't know.
Then you write Haskell, and have to use hdmake - which of course is written as a haskell program, using a fancy monad you don't know, and python has to use pydmake, and ruby has to use rbdmake, and scheme has to use sdmake, and lisp has to use ldmake, and asm has to use 60 different dmakes, depending on which asm you're using.
Instead, we all use make. Make allows for arbitrary code to be executed, so no matter what programming environment you use, you can use a familiar build tool that everybody knows. Sure, java has Ant, Jelly, Gradle and god knows what else, and node has $NODE_BUILD_SYSTEM_OF_THE_WEEK, but even there, you can still use make.
That's the power of generic tools.
Let's try it then. The declarative, build system has a formal spec with types, files, modules, ways of describing their connections, platform-specific definitions, and so on. Enough to cover whatever systems while also being decidable during analysis. There's also a defined ordering of operations on these things kind of like how Prolog has unification or old expert systems had RETE. This spec could even be implemented in a reference implementation in a high-level language & test suite. Then, each implementation you mention, from rdmake to hdmake, is coded and tested against that specification for functional equivalence. We now have a simple DSL for builds that checks them for many errors and automagically handles them on any platform. Might even include versioning with rollback in case anything breaks due to inevitable problems. A higher-assurance version of something like this:
https://nixos.org/nixos/about.html
Instead, we all use make. Make allows for arbitrary code and configurations to be executed, so no matter what configuration problems you have, we can all use a familiar build tool that everybody knows. That's the power of generic, unsafe tools following Worse is Better approach. Gives us great threads like this. :)
>You could've just as easily said the common, subset of SQL could be implemented extremely different in SQL Server, Oracle, Postgres, etc. Therefore, declarative SQL has no advantages over imperative, C API's for database engines. Funny stuff.
No, that's not my point, my point is that a build tool that meets parent's requirements would necessarily be non-generic, and that such a tool would suffer as a result.
>Instead, we all use make. Make allows for arbitrary code and configurations to be executed, so no matter what configuration problems you have, we can all use a familiar build tool that everybody knows. That's the power of generic, unsafe tools following Worse is Better approach. Gives us great threads like this. :)
Worse is Better has nothing to do with this. Really. Make is very Worse is Better in its implementation, but the idea of generic vs. non-generic build systems, which is what we're discussing, is entirely orthogonal to Worse is Better. If you disagree, I'd reccomend rereading Gabriel's paper (that being Lisp, The Good News, The Bad News, And How to Win Big, for the uninitiated). I'll never say that I'm 100% sure that I'm right, but I just reread it, and I'm pretty sure.
A build system is essentially supposed to take a list of things, check dependencies, do any platform-specific substitutions, build them in a certain order with specific tools, and output the result. Declarative languages handle more complicated things than that. Here's some examples:
https://cs.nyu.edu/~soule/DQE_pt1.pdf
I also already listed one (Nix) that handles a Linux distro. So, it's not theory so much as how much more remains to be solved/improved and if methods like in the link can cover it. What specific problems building applications do you think an imperative approach can handle that something like Nix or stuff in PDF can't?
"It's fully generic."
It might help if you define what you mean by "generic." You keep using that word. I believe declarative models handle... generic... builds given you can describe about any of them with suitable language. I think imperative models also handle them. To me, it's irrelevant: issue being declarative has benefits & can work to replace existing build systems.
So, what's your definition of generic here? Why do declarative models not have it in this domain? And what else do declarative models w/ imperative plugins/IO-functions not have for building apps that full, imperative model (incl make) does better? Get to specific objections so I can decide whether to drop declarative model for build systems or find answers/improvements to stated deficiencies.
http://www.cs.vu.nl//~kielmann/papers/THD-SP-1991-04.pdf
https://github.com/cmungall/plmake
Add to that Myreen et al's work extracting provers, machine code and hardware from HOL specs + FLINT team doing formal verification of OS-stuff (incl interrupts & I/O) + seL4/Verisoft doing kernels/OS's to find declarative, logic part could go from Nix-style tool down to logic-style make down to reactive kernel, drivers, machine code, and CPU itself. Only thing doing arbitrary execution, as opposed to arbitrary specs/logic, in such a model is what runs first tool extracting the CPU handed off to fab (ignoring non-digital components or PCB). Everything else done in logic with checks done automatically, configs/actions/code generated deterministically from declarative input, and final values extracted to checked data/code/transistors.
Hows that? Am I getting closer to replacing arbitrary make's? ;)
Each filetype is accepted by a program. That program is what we'll want to use to compile or otherwise munge that file. So, in a file somewhere in the build, we put:
*.c:$CC %f %a:-Wall
*.o:$CC %f %a:-Wall
The actual DMakefile looks like this:
foo:foo.c:-o foo
bar.o:bar.c:-c
baz.o:baz.c:-c
quux:bar.o baz.o:-o quux
all:foo quux
This is something I came up with on the spot, there are certainly holes in it, but something like that could declaritivise the build process. However, this doesn't cover things like cleaning the build environment. Although this could be achieved by removing the resultant files of all targets, which could be determined automatically...
Far as what I was doing, I was just showing they'd done logical, correct-by-construction, generated code for everything in the stack up to OS plus someone had a Prolog make. That meant about the whole thing could be done declaratively and/or c-by-c with result extracted with basically no handwritten or arbitrary code. That's the theory based on worked examples. A clean, integration obviously doesn't exist. The Prolog make looked relatively easy, though. Mercury language make it even easier/safer.
One of the guys working on Redox OS make a library called cake which provides macros for a Makefile style script: https://github.com/ticki/cake
M4 and sh are very concise languages. Nonetheless autotools is orders of magnitude more complex than Maven. You really can't compare at all.
At any rate, if you want a more concise syntax there is gradle (but it's a bit slower as it's actually executing a real scripting language) and, perhaps a nice middle ground, a thing called Polyglot Maven which is the same build engine but with a variety of non-XML syntaxes. The YAML one is quite nice:
https://github.com/takari/polyglot-maven/blob/master/poms/po...
That way you get simple syntax and a simple model, but still with lots of features.
Building software is a programmatic process. No XML please! We're decidedly not on Windows, and since I have the misfortune of fitting such square pegs into round holes, please don't use XML for applications which must run on UNIX. It's a nightmare. It's horrible. No!!!
Of course, like any real programming language, it's hard to process with regex, but then again, I don't want to process makefiles with regex. And you might have some luck coaxing AWK or the SNOBOL family to parse it, and it would be far easier than doing the same with XML.
>please don't use XML for applications which must run on UNIX. It's a nightmare. It's horrible. No!!!
I'd disagree with you there. DocBook, HTML, and friends, are all good applications of XML (or near XML), doing what XML was designed for: Document Markup.
Seriously people, when you're writing a program in a language that has "Markup Language" in the name, does that not ring any alarm bells?
Yes, I said JSON. JSON is very easy to parse, and you can grab unique key/values, which are most of them, with this regex:
/(,|\{)\w*\"<key>\"\w*:\w*(.*?)\w*(,|\})/As for when your record spans multiple lines, with recursive structures, the previous regex is for extracting simple atomic data from a json file, which is usally what you want in these cases anyway. If not, the json(1) utility can, I believe, extract arbitrary fields, and composes well with awk, grep, etc.
Also, because JSON is so common, you get really good tooling for handling structured data by defult, instead of kinda-okay tooling for 50 different slightly-incompatable formats. 10 operations on 10 datastructures vs 100 operations on 1, and all that.
But for unstructured data, or for one-level key/value data, JSON is overkill. You can use DSV, like this:
key1:value1
key2:value2
and so:onYAML doesn't need any special tools - it's ASCII and can easily be processed with AWK, for example.
I don't know about you, but the last thing I want is to have to have a whole new set of specialized tools, just so somebody could masturbate in XML and JSON.
XML is a markup language. That means it's for documents, possibly for documents with pictures, perhaps even with audio. It's not and never was meant for storing configuration or data inside of it. XML is designed to be used in tandem with XSLT, and XSLT's purpose is to transform the source XML document into (multiple) target(s): ASCII, ISO 9660, audio, image, PDF, HTML, whatever one writes as the transformation rules in the XSLT file. XML was never meant to be used standalone.
If you really want to put the configuration into an XML file, fine, but then write an XSLT stylesheet which generates a plain ASCII .cf or .conf file, so its processing and parsing can be simple afterwards. XML goes against the core UNIX tenet: keep it simple.
Do you like complex things? I do not, and life is too short.