Hacker News Reader

Comment by sansnomme | Hacker News Reader

Package managers need to automatically derive properties of end builds based on licenses. E.g. Eclipse License 2.0 without the presence of another more liberal license means it cannot be used in copyleft software, any dependency that is copyleft is also infectious etc. Of course it won't account for every single legal property but the basic checks should be done. To prevent work duplication a single binary/library written in Go/Rust can take care of this problem and be used across all package managers. It is in the interest of GitHub to sponsor such a project.

View on HN

This is one of the services that distros offer, for what it's worth. Every package in Debian is checked for copyright compliance against the DFSG, and annotated with the licenses that are controlling for that code.

This is a reason for Debian to exists.

Licenses cannot be reviewed automatically in a reliable way. Debian developers review the licenses and store them in a machine-parsable file.

Are you saying that standard licenses cannot be checked for compatibility automatically? Or that arbitrary homebrew licenses cannot be checked automatically?

Didn't Debian push a stable security update for Ghostscript that changed the license over to AGPL some time ago?

Did they break the license meta data while doing so?

There is https://www.fossology.org/ which helps a bit.

There are a lot of tools in this space:

https://wiki.debian.org/CopyrightReviewTools

If you want to do license compliance for various package managers then I recommend you checkout https://github.com/heremaps/oss-review-toolkit.

Full disclosure: I am one of the maintainers of OSS Review Toolkit.

Agreed. I found this project helpful for automating dependency checks: https://github.com/pivotal/LicenseFinder

I actually played around with turning it into a GitHub action for further usability improvement, although it needs more work: https://github.com/ralexander-phi/license_approval

Pivotal Lab’s license finder used to be my go to recommendation (and still is if “free” is an absolute requirement), but if a company takes license management seriously, I’d recommend checking out FOSSA[0] instead. It’s significantly better and fairly reasonably priced.

[0] https://fossa.com/

The licensing of a dependency could be easily determined programmatically (GitHub already built a decent scanner). However, I think that the quality of a dependency is most important and that requires a manual vetting process. A trivial solution would be to create a crowd-sourced dependency vetting platform.

GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.

Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere?

It makes license scanning a doddle.

> It makes license scanning a doddle.

As someone working in the field of license compliance (leading an Open Source Program Office) and dealing with the licensing of tens of thousands of OSS dependencies on a daily basis for a large company I can tell you that license scanning / compliance is anything but a doddle. I do a lot of public speaking on this topic for a summary of the issues I recommend you to have a look at https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To...

Most modern package managers do offer project maintainers a way to declare the license for a project. However I can tell you that often the declared license does not match the licenses detected in the source code. What counts is the license stated in the source code files not what it's in the gemspec, package.json, pom.xml etc.

This is quite common issue especially for older or larger OSS projects where various contributors have added new code over time that may be licensed under an OSS license that is compatible but not the same as the main license of the projects (Think adding BSD-2-Clause in Apache-2.0 project) What happens is that this contributions get accepted but the project maintainers do not to update the declared license.

Not saying declaring a licensing in a .gemspec file is useless but just recommend you to take it as an indicator of the main license of the project - the project may include source code that licensed under a different license.

Lack of clarity around licenses and security vulnerabilities is a big issue within the OSS community especially as lack of clarity reduces engagement — that means fewer users, fewer contributors and a smaller community. Several organizations are working on a open source solution for this community problems see for further details https://clearlydefined.io/about

Full disclosure I am one of the maintainer of OSS Review Toolkit and contributor to ClearlyDefined.io

An even older tradition is a LICENSE file in the root of your project, which GitHub seems to read, but it's not great at matching up the license text to actual licenses.

An alternative tradition uses a COPYING file. I'm not sure which is older, but most GNU projects use COPYING instead of LICENSE. Meanwhile, some projects use COPYRIGHT or other variation, BSD-style licenses are often added directly to source files, and properly applying LGPLv3 to your project involves adding a separate COPYING.LESSER file. It's a mess.

I think most modern package managers have something like that: NPM has a license field in package.json, pypi in pkg-info, nuget in the nuspec file, etc.

Github uses the LICENSE file to detect licenses by default IIRC.

If you trust the license listed in gemspecs alone, you are almost guaranteed to be in violation of a license somewhere. ;)

It also can get things problematically wrong. Like licenses based on BSD3 with extra clauses tend to show up as BSD3.

But people will trust the license tag, and end up breaching the license.

Well, Linux distros are already pretty much that - crow-sourced dependency vetting platforms. Also take care of the bits actually fitting together & quite a bit of QA.

And so very old.

if you want the newest and shiniest there are rolling release distros.

> A trivial solution would be to create a crowd-sourced dependency vetting platform.

And then aren't you just recreating the old fashioned Linux distribution? No-one writes webapps to Linux distros any more; they're always based on language-specific, author-submitted, untrusted package managers because waiting for enough trust to build up to include the latest version of unnecessary-wrapper-for-document.getElementById-0.83.2 is considered stifling.

Maybe nixpkgs comes a little close, somewhat trusted and requiring third-party involvement, multiple versions simultaneously, but including new packages quickly enough. (And indeed, nixpkgs acts as distribution that can run on top of your distro or standalone as NixOS.)

> A trivial solution would be to create a crowd-sourced dependency vetting platform.

Such a platform is already being developed by various organizations, see https://clearlydefined.io/about and it already in use by GitHub. It's still under development and the initial focus is sharing data regarding copyrights, licenses and source code location for OSS packages.

Note that ClearyDefined only scans the code repository of OSS project it does not resolve dependencies for scanned OSS project. This makes sense as not all dependencies of OSS project B are inherited by a project A which included B as a dependency (due to dependency version resolution by the package manager or dependencies only used for testing).

The idea is that you will us an tool to resolve dependencies for the package manager your project uses which then queries the ClearyDefined APIs.

We are building such a tool as an open source project to manage of licensing and security for dependencies named OSS Review Toolkit (https://github.com/heremaps/oss-review-toolkit).

For a overview of the solution we are building see slide 9 in https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To...

Full disclosure: I am one of the maintainers of OSS Review Toolkit and also a contributor to ClearlyDefined.

I think each respective package manager should just support this out of the box.

Reply on news.ycombinator.com