Reggae: A build system with multiple front ends (D, Python) and back ends | Hacker News Reader

Reggae: A build system with multiple front ends (D, Python) and back ends

github.com

76 pointsolvy017 comments

I love the idea of a build system separated into front ends and back ends. Imagine if CMake were just an LLVM-esque backend and you could use a nice Python (or Lua or <insert your favorite language>) frontend instead of the awful proprietary CMakeLists frontend.

You can go a step further and factor the build system core into a library and provide dynamically buildable/loadable modules. This way you can reuse the same core in several drivers (different frontends, build-by-convention like Cargo, etc) or embed it into IDEs. This is what we did in build2[1][2].

[1]https://build2.org/release/0.12.0.xhtml#build-system-modules

[2]https://build2.org/release/0.12.0.xhtml#library-context

I don't want to nitpick too much, but there is nothing particularly proprietary about the CMake front-end. It's as open source as Python. It's just less common.

Yeah, proprietary was probably the wrong word. But it's basically a scripting language you have to learn specifically for cmake and can't reuse anywhere else.

I would not call it frontend and backend, but build configuration and build execution.

For execution Ninja is nearly perfect. For configuration the needs differ a lot.

Disagree. If you design your framework with language-portability in mind, you're going to have to trade off on a lot of other things you care about. You'll have to go with lowest-common-denominator design-decisions, rather than making a language with its own idioms.

LLVM itself, for instance, is a C++ API which also exposes a C API. People generally favour the C++ API. Idiomatic C++ is nicer to work with.

There's plenty wrong with CMake, but the lack of Python bindings isn't the problem. CMake's scripting language is clunky and error-prone, with documentation that is very complete, but surprisingly unhelpful for doing real work. That's the problem. The lack of choice isn't.

CMake gets all the hard stuff right. I really don't think it would be that hard to overhaul its usability, but I doubt it will happen.

> CMake's scripting language is clunky and error-prone, with documentation that is very complete, but surprisingly unhelpful for doing real work. That's the problem.

And wouldn't one solution be to use an alternate front-end with less clunky and error-prone syntax (say, python) that has doc strings so you can hover over an API in your IDE and read what it does, get auto-complete, etc.?

    add_library(foo SHARED
        foo.cpp
        bar.cpp
    )

vs.

    import cmake

    cmake.add_library(name="foo", shared=True, sources=[
        "foo.cpp",
        "bar.cpp",
    ])

People know how to do stuff in python. Stuff like loops and if statements and string operations. Nobody knows how to do that in CMake. The benefits of allowing arbitrary language bindings are immense.

> And wouldn't one solution be to use an alternate front-end with less clunky and error-prone syntax (say, python) that has doc strings so you can hover over an API in your IDE and read what it does, get auto-complete, etc.?

You're right of course that there are great advantages to using existing languages with familiar syntax and good existing tools. I agree that CMake should have gone with an existing language rather than inventing a bad one of their own. Same applies more generally to projects that roll their own language for no clear reason, such as FlightGear. [0]

For me the question is how much work it would be. If they could introduce a decent Python frontend, in addition to supporting their existing language, I agree it could improve things greatly.

> The benefits of allowing arbitrary language bindings are immense.

Arbitrary languages? If that's the end-goal, presumably they'd really just be exposing a C API (the way LLVM does), or a machine-friendly text interface (LLVM supports this approach too with its own assembly language). I don't think that would be the way to go, but I can see the case for supporting one language well. Python3 would be a fine choice.

[0] http://wiki.flightgear.org/Nasal_scripting_language

This is a pretty fascinating idea! I definitely like the build systems that embed a DSL in a real programming language, like SCons (Python), Rake (Ruby), or Gradle (Groovy). Reggae lets any language have a build system.

Further, Reggae has multiple backends, just like CMake. And the generated build scripts can be shipped without Reggae, just like how Autotools works. Neat!

This is misguided. If you look at any large cmake build, the cmake language is not the source of problems. It's the expressivity of the language that's the source of problems. Without rigorous style guides, developers can get as wild as they want. Cmake builds launching other cmake builds as part of the build? Sure throw that in! Loops, predicates, globals of unknown definition? Why not! Cyclomatic complexity run amok.

Choosing general purpose languages like D or python can only compound this problem. As anyone who has worked with Scons will tell you, having a fully functional language at your fingertips is a recipe for an opaque, undebuggable build.

The build syntax must be expressive as necessary, _but no more_.

> Pick a language to write your description in and place a file called reggaefile.{d,py,rb,js,lua} at the root of your project.

This sounds like a nightmare, not a feature. It's an interesting technical experiment, to be sure. But not a production structure.

Which is a big reason why I dislike Gradle, a slower version of Ant, with more spaghetti build code and memory usage.

As someone who "gets wild" in cmake, if it did not allow it I would basically have to write my own build system to do what I want (mostly parsing / preprocessing / code generation). These steps have to be somewhere and I'd much rather reuse the infrastructure available in cmake for that rather than roll my own - especially given how easily it runs on windows.

I've also had to get wild in cmake (codegen), but in hindsight, the code that I wrote for that was the most brittle part of our build system at the time.

I'd love to know of an alternative to writing gnarly cmake for stuff like this though, particularly to work cross platform and generate IDE projects.

While the idea of using a common build backend is nice, I also like the idea of a build DSL common to all projects, rather than each project using their binding of choice (and associated idioms) for the build files.

Build error, build failing

https://imgur.com/mL9ILZO

This comment isn't very useful or interesting.

How people use CI varies wildly by project, and only sometimes has a useful signal.

For example, this project uses appveyor (where the build is passing, but the badge is wrong), and travis-ci with 7 targets (where the build is passing on 5 targets, and failing on 2 due to errors from dub (d's package manager), which the author did not write).

The author's code is not causing those CI failures, and if anything the failures reflect poorly on travis-ci and appveyor since both are on their end.

Why put the badge on the readme then?

Reply on news.ycombinator.com