Please – A Cross-Language Build System
please.build
please.build
So it isn't general purpose?
I had some tests that needed a binary file. That binary file was generated from a text file by a utility program. That utility program had to be built from source.
After a week of trying to get that correctly specified, I gave up. I thought I'd found the right way of doing it several times, but each time I discovered the dependency chain was broken in some way. I looked into how official modules that dealt with generated files were implemented, but I found them to be surprisingly complex.
By comparison, it was only mildly annoying to do that sort of thing in CMake (the build system I moved to after a year or two with waf). The relative simplicity of CMake was what convinced me to switch.
That being said, it's been several years since I last used waf. It may have improved during that time.
The docs are up front that waf is not a build system but a build system toolkit. For example, there is a demo that uses makefiles as the build language and runs them in waf. Another demo requires no build file but just finds the source files and builds them. But I have yet to see any external project released that says, “This is a build system... built on the waf build system toolkit,” like the Software Construction Toolkit was built on scons. Perhaps the toolkit is so close to being a build system that people just use it as is and require only small customizations, which are kept in their projects.
redo: build rules spread across many files
There are other projects that use the bitbake build system, not just OpenEmbedded and the Yocto Project.
"Stringly typed" - even lists are just semicolon-separated strings.
Insane quoting rules. I recently tried to work out exactly how the argument parsing works and the source code has undocumented behaviour. It's ridiculous.
Accessing undeclared variables works fine. They're just empty. Good luck tracking down silently ignored typos!
Scratching the surface...
Having said that, there might be a way. From their FAQ:
> It might well be possible to make it work using Cygwin or MinGW / MSYS, or the recent Ubuntu on Windows development though. We aren't doing this work ourselves because we don't have any Windows machines, but we're happy to accept PRs in that direction.
Not sure I want to use a platform which the dev team can't test (and there are other alternatives), but it might work after all.
A side remark, the link posted below in this thread contains an awesome "awesome" list. For those interested, here is a (truly) awesome list of dataset resources: https://github.com/awesomedata/awesome-public-datasets.
I encourage Windows versions of cross-platform apps wherever the app could justify it due to all the developers and tooling on Windows. Yet, non-Windows apps can be cross-platform.
Just offering a counterviewpoint.
> Shake is a library for writing build systems. Most large projects have a custom-written build system, and developers working on the project are likely to run the build system many times a day, spending a noticeable amount of time waiting for the build system. This document explains why you might pick Shake over alternative tools for writing build systems (e.g. make, Ant, Scons).
Shake is written in Haskell and is open source.
Premake was promising but I fear it fell victim to early hype syndrome.
Fuse support on Windows is a no-go, very unstable, broken, and unavailable by default.
makefiles without any magic are comprehensible, but people quickly add magic :(
https://www.gnu.org/software/make/manual/html_node/Catalogue...
If you only use plain rules and maybe simple pattern rules, avoid configure step, etc. Makefiles are simple, pretty and very generic.
Why do all other build systems have so much magic?
Why not declare rules and input/output in a pure functional manner?
While my experience of `plz` may have been mired by the project that used it, I can't say I found it added anything special.
It arguably doesn't meet the needs of the `plz` developers themselves, as evidenced by the custom build-rules in the project that overrode the built-ins. And at one point, it was suggested to us that we drop Plz and move to buck/bazel/pants which was suggested would be a relatively simple replacement.
Sure, it built stuff, but the general impression was it got in the way, while due to the way it was configured, was essential to keeping the project running.
And why are there 20 releases on the 25th of November?
I'm glad I'm not the only one with this problem :P
What's the advantage of using Please?
A self-contained binary has everything it needs to function and is typically much smaller. If I’m not using Java for my work, then I don’t want to be encumbered with having to think about it, or stay up to speed with releases and everything else.
I like to see projects with as few dependencies as possible, even when statically linked. Bringing in the JDK when it’s not needed is tiresome.
ruby on the other hand gets quite a bit of flack as well, because most distributions are so far behind the official ones. that's probably why an embedded ruby binary has become a thing.
Especially virtualenvs are an axe I like to grind.
I see no way in which this "leaves your computer at their mercy" more than any other process of purposefully executing code they control on your system.
I would caution you to use the phrase “no way in which” when discussing security - the less informed may read this and believe it.
While an edge case and requiring a mailicious targeted attack in this case there’s at least the possibility of being MiTM’d.
The problem - as you’re probably aware - with using absolute terms when speaking about a case like this is that it’s easy to extrapolate this sense of safety to something that may lead to an attack that requires much less precision thank convincing the browser that the MiTM proxy is “please.build”.
For the inner downloads in the script, they use the -fsSL flags, which would protect against such broken behaviour. But not their use-facing script.
More to the point, the install just downloads:
https://get.please.build/${GOOS}_amd64/${VERSION}/please_${V...
then unzips and links it to PATH. No checking the source isn't corrupt, no checking if the tar archive successfully expands. (And the var GOOS seems to depend on an environment variable I don't think is guaranteed to exist. It certainly doesn't on my Mac.)
If that's the case... Why not just provide a download link? It won't have the same issue as a broken install if the connection drops, and is just as easy. The only technical bit, linking to PATH, is something the end audience could be expected to know.
What they should be doing is to understand what's actually going on.
Once you download and run software from a TLS enabled website, you're putting trust in that website. It doesn't really matter if you are doing 'curl https://example.com | bash' or downloading a binary. They can't be MITMed any more so with the curl way than downloading a binary.
That's all there is to it. I realize there are many security people who advice against doing the curl thing, but I feel people should realize it is a rule of thumb with unsecured websites.
It’s sometimes easy to forget that there are various reasons people are less informed. For people early in the career for example, reading security advice on HN and believing it is not unheard of especially when you have real experts on here who know what they’re talking about.
Something that is “all there is to it” for you is not necessarily the same for someone else. I’m not saying it’s anyones job to inform them; just that it’s better not to speak in absolutes.
If security folks say something, I assume there’s a good reason until I become a security folk myself - it’s one of those fields where being sceptical about everything third-party helps.
My point really wasn’t about the MITM but a there’s always a possibility that a proxy on a public/compromised network can intervene between you and a “secured” website.
Binaries can be verified with checksums to make sure that the artifact hosted on the repository is indeed what you received. You are 100% correct that you are still trusting the code from the third party developer though!
Same can happen to a binary.
>Binaries can be verified with checksums to make sure that the artifact hosted on the repository is indeed what you received
Here's the problem. The checksum is usually on the same page the binary is located. Which pretty much defeats the purpose.
I guess the only threat there (that I can see) is that the install script can be malicious, but your point was that you’re trusting the owners of the website anyway by downloading their binary and executing their code locally. It can be argued that it’s probably easier to ship malicious code outside the main repository (e.g. in an install script) but I do not have a good counter besides this weak argument.
The checksum is indeed usually on the same page and does make it useless in my hypothetical MITM.
in any event, it's more surface area. their web server being compromised and serving a bad shell script is just more that can go wrong.
If they were serving up a binary you would have the same exact threat that you mentioned.
The threat model barely, barely changes when talking about curl | sh vs downloading and manually executing a binary. Barely.
We already have a serious incompatibility problem with projects using autotools vs CMake vs Meson vs gyp vs Boost.Build vs SCons vs BUCK vs... and now we throw Please onto the pile.
It sucks when you find a smallish library and discover it uses an esoteric build system whose dependencies dwarf the library themselves (cough Yoga). The OSS community needs build system consolidation, not tacking on a 15th wheel to the cart.
Read the FAQ, it's just bananas:
> It [Bazel] is a great system but we have slightly different goals
> We preferred [Buck] to other options available, but again we're focused on different goals
> we didn't think it [Pants] was the ideal fit for us at the time
"All other systems had slightly different goals or weren't the exactly ideal fit, so rather than contribute to or extend them we poured an enormous amount of energy into rebuilding them in a slightly different way."
This is awful demanding. Not entirely unexpected in tech tho
“Why don’t others fix my frustration in open source projects!” they shout with no hint of irony
Consider that the parent comment author might already be making pull requests to the projects of his interest. The snarky "PRs welcome" is therefore unnecessary and unwelcome. Everyone here knows PRs are welcome in an open source project. Some of us do send PRs. But whether someone sends PRs or not has no bearing on whether one can appreciate or criticize an idea as long as it is done in a substantive manner.
In fact, whether the parent comment author sends PRs or not is orthogonal to his complaint that we have too many build systems that is increasing the burden on users and maintainers. If you have something to say against this point, please do so on its own merit in a substantive manner without insinuations or ad-hominem attacks.
Maybe Please is so dramatically better that it pulls tons of projects into its orbit! What a great outcome: I'd contribute PRs for every missing use case, accelerating the consolidation.
But if (as the site says) the goal is building something at parity but with a slightly different focus, then it can only produce further fragmentation. Projects that adopt Please inject dependencies on random S3 buckets and .bashrc edits. I can't accept that in the software I maintain, so any Please-built projects are de-facto inaccessible. That's bad.
By all means do what's best for ThoughtMachine, but publishing and promoting this makes building OS software harder, not easier.
Meson is fantastic.
The binary it produces is still dependency free.
Distributing binaries outside of package managers is a) a rarity in the Linux world, and b) not in keeping with the open source philosophy. Requiring end users to have python so they can compile a c/c++ application will never make sense for a large portion of the user base build systems target.
And like I said, many Linux systems (and almost any used as a developers machine) will have Python installed by default.
And the simplicity of meson's build files far outweighs any issues installing it.
But the project now depends on Python, pip and Meson.
I mean it's not like Python is a rarity on Linux systems, or a difficult thing to install (assuming it wasn't installed by default)
What sucks more to me than finding a lib using its own build system is finding a lib that won't adopt more modern ones or languages that refuse to fix their defacto language-specific build systems, both caused by the sentiments you state about having too many and/or it being too hard for builders to change their ways. There is a middle ground here and it starts with asking software to move forward, not asking it to stop.
I don't think it's that simple. Sure, I could spend time figuring out the GNU Make source code. I could somehow shove a C preprocessor into it and make it automatically generate dependencies based on include directives. I could make it depend on clang. Will the make maintainers accept my patch if I send it their way, though? Somehow I doubt it. What if I changed the make language to make it easy and unambiguous to write paths with spaces and other problematic characters in them? Do you think the maintainers are going to accept a patch that will most definitely break existing makefiles that work perfectly fine just because I fixed a major limitation of the program? We're talking about a tool whose grammar requires that commands be prefixed with a tab just because the author experimented with that rule and then couldn't change it because about a dozen friends were already using it. Fixing make is simply not possible at this point. If you try, it will just become yet another incompatible tool, some kind of make derivative that will have to be maintained independently. At this point, why limit yourself to make? Might as well rethink the whole thing.
As far as GNU/Linux distributions go, make and autotools are the default. If you're using anything else, it's gonna be a build time dependency that people are going to have to install in order to build your program. So you look at the other tools and you find they don't quite do what you want either. They only do about 80% of what you want, and getting it to do the remaining 20% makes you really wish the tool was an actual programming language instead of a limited build script. Is it any wonder people roll their own?
Also, I believe people should work on what they personally like. There is no obligation to contribute to some project just because it's "standard". It's an interesting problem that's difficult to solve, so it's not weird to see lots of people offering their executable opinions on how things should be done. If programmers don't like what's available out there, they are welcome to try and come up with something better. Who knows what will happen? Maybe the new wheel will turn out to be better than the current one. Maybe it won't. It might introduce new ideas that will influence new designs. These ideas might even make their way into other systems.
But the C compilers already provide tools for example `gcc -MMD`
This couls also be said about many other things as well. For example package managers and test suites. Even worse is that every language feels like they are obligated to build their own because they are obviously the best programmers on the planet (them using that language is a proof of that obviously). What results is dozens upon dozens restrictive and sub-par build systems, package mangagers and test suite that are all used by only a small group of people and are just an extra dependency for everyone else.
>It sucks when you find a smallish library and discover it uses an esoteric build system whose dependencies dwarf the library themselves (cough Yoga). The OSS community needs build system consolidation, not tacking on a 15th wheel to the cart.
Developers generally don't think of the build system as a dependency because tehy already have it installed on their own machine. Though it shoudl also be mentioned that a big portion of developers (especially those using languages that encourage to install everything with their own package manager) have a very low or no barrier for adding new dependencies.
My project, as well as many other projects, ships its own default.nix for builds. Linking an example would deanonymize this account a little too much, but there's plenty of examples out there.
I’ve used Buck; it’s slow. I haven’t tried Pants.
I agree with this guy. It shouldn't be that hard to include an example of an advanced script.
https://www.gnu.org/software/make/manual/make.html#toc-Compl...
Also, make is very barebones and "close to the metal". Good modern build systems provide libraries of common tasks so you don't have to reinvent the wheel; they can handle parallelism well; and can fall back on makefiles where needed.
They let you get more done in less time.
- the archaic custom scripting language (that's its main problem)
- there are many ways to do the same thing, resulting in each (non-trivial) build script looking different
- for the above reason, importing dependencies written by somebody else is non-trivial, and its often better to rewrite the cmake scripts completely
- it lacks some 'integration features' found in more modern systems like Rust's cargo: (1) proper dependency/package management (2) run targets (3) switching between build configurations should be easier (not just release/debug, also different build target platforms)
cmake pros (also IMHO):
- can generate IDE project files (that's what most new build systems completely ignore)
- very good cross-compilation support
- describing a simple build is reasonably simple
- broad support for build tools and IDEs
CMake is firmly in the "filenames don't contain spaces" era, along with make, Autotools etc. Granted it is more or less the standard at the moment but look at Meson to see how totally insane it is.