Hacker News Reader

Comment by g-b-r | Hacker News Reader

> First, I'm uncomfortable with making a package creator's VCS provider part of the language's module infrastructure.

Why can't you store a (possibly shallow) clone of the repository in the "package" registry?

> Second, not all files under version control necessarily belong in a tarball

How likely is it that files under version control but "not belonging in a tarball" (not needed to compile the current version) take so much space to justify making a tarball? At least if you make a shallow clone.

> not all files in a tarball necessarily need to be under version control

Which wouldn't (besides huge files that can be handled with git-lfs and similar)?

If you have configure in mind, let's not do it please.

The risks introduced by including them vastly overcome their convenience, in my opinion.

And maybe let's move towards getting rid of autotools

View on HN

Why can't you store a (possibly shallow) clone of the repository in the "package" registry?

Yes, that would be an option.

As to your question, you generally want to version control input files, but distribute the generated artifacts. Some possible scenarios would be documentation, data tables in source or binary form, generated code including maybe even configure scripts if you cannot avoid it (though if I had to 'vendor' 3rdparty code that uses autotools, I think I probably would just unpack a tarball and commit the relevant artifacts).

If you distribute generated artifacts you're not really distributing the source (and thus xz can happen)

Even for documentation, you'd think it would be harmless to generate it, but it's far from a given that the tools (e.g. browsers) used to read it are safe from malicious documentation files

it's far from a given that the tools (e.g. browsers) used to read it are safe from malicious documentation files

Generated HTML files are potentially easier to audit than the scripts/toolchains used to generate them on an end user's machine if you do not pre-generate them.

Off the top of my head, other things I've done is committing RELAX NG *.rnc files, but shipping *.rng files, or generating C header files for various types of data (think `xxd -i` in case of binary files, but also just large chunks of plain text that gets wrapped into a C string).

Hmm, potentially, the problem is that hardly anyone ever audits packages, while at least someone occasionally gives a look at a repository history

Can't the things you list be part of the build scripts?

Sure - but running build scripts on an end-user's machine requires the user to have all relevant tools installed, and isn't exactly reducing the attack surface...

I still like the idea of shipping tarballs that include generated files instead of pulling input files from source control. As mentioned, the first thing that came to mind to make things easier to audit is to stick their contents into a community-controlled VCS.

> Sure - but running build scripts on an end-user's machine requires the user to have all relevant tools installed, and isn't exactly reducing the attack surface...

Well, that's what building from source means; you could maybe distribute some compiled files for those who prefer them, and are willing to take the risk.

But you seem to be arguing that having users compile their software themselves doesn't increase their security?

> As mentioned, the first thing that came to mind to make things easier to audit is to stick their contents into a community-controlled VCS.

True

But you seem to be arguing that having users compile their software themselves doesn't increase their security?

On the contrary! I gave examples where source files get generated. This happens in one of two places: Either when the maintainer publishes a new version, or every time an end user builds the package.

I'm arguing for the former, you're arguing for the latter. There are pros and cons to either approach. Some cons for the latter:

First, the build process becomes more brittle, as every end user now has to install the necessary tools. In case of the project containing RELAX NG schemas I alluded to, this would require a recent version of Perl, a Java runtime, and the Trang utility written in Java. The alternative? Just shipping a single XML file.

Second, the build process often doesn't become more, but less auditable: Instead of just reviewing the source files that actually get compiled, you now have to track down how they get generated, and review all the scripts that do so.

Reply on news.ycombinator.com