Bringing supply chain security features to the Go community | Hacker News Reader

Bringing supply chain security features to the Go community

github.blog

183 pointslmas71 comments

GitHub's dependabot is causing a ton of "spam" in our frontend (Angular) repositories, as it seemingly opens 1-5 PRs per day to bump random dependencies. I really hope this does not become common practice for our Go repositories.

Is there an option to tell dependabot "make one PR per week at most, please, and bundle your changes"?

Part of that is the Node / JS ecosystem that churns out libraries for every tiny thing. The Go ecosystem is a lot more streamlined; adding dependencies is frowned upon, building small libraries is frowned upon, etc.

That said, if it's just version bumps then they should be bundled. But what about security issues? Those need to be fixed and updated asap.

Here's some information on these "vulnerabilities": https://overreacted.io/npm-audit-broken-by-design/

As far as I'm concerned, there's no need for Dependabot to create PRs. The notifications in the security tab are enough. Mark the unnecessary ones as benign.

The PRs are really helpful when you do actually want to update.

You can have Dependabot create a PR by hitting a button somewhere in the vulnerability details.

And here's the HN discussion on that thought-provoking post:

https://news.ycombinator.com/item?id=27761334

The amount of pull requests from dependency updates can become more manageable if you enable GitHub’s Auto Merge feature[1]

[1] https://docs.github.com/en/github/collaborating-with-pull-re...

Exactly what you want to have: get a bot merge fixes for security issues prepared by another bot without any human oversight. Then let Copilot add features to your software automatically as well.

You can't enable granular auto merges: I'm fine with auto merging minor/security update PRs from dependabot. Not anything else. You can't set up that.

You could probably use a GitHub Action and check the API for author of the pull request if you want to require additional verification. If have done something similar with auto accepting docs-only pull requests in the past.

Dependabot PM here:

We've provided an action that will provide metadata (e.g. semver bump) that you can use to enable granular automerge. See https://github.com/dependabot/fetch-metadata/#enabling-auto-... for an example.

This seems like good motivation to prune your dependencies if they’re getting out of hand. Of course, easier said than done, but at an ecosystem level maybe it will help?

Well angular repositories tend to have tons of dependencies.

In the case of NPM, the only way to not have your dependencies be “out of hand”, is to not use NPM.

/s (but also, for realsies)

Disable it. If you don’t want to, limit it to weekly or monthly. I agree that Dependabot is super noisy especially by default.

Yup, there's quite a few different configuration options available: https://docs.github.com/en/code-security/supply-chain-securi...

The "biggest" problem in the JavaScript ecosystem (NPM) is that dependadabot doesn't know how to discriminate between dependancies and devDepandancies.

I don't really care if jest includes a package that has a regex issue. It's not production code. I do care if babel introduces a backdoor, but somehow they're treated with equal importance.

It depends, attacks via build pipelines can be devastating.

Yep.

Unfortunately it’s not clear cut that we don’t have to worry about devDependencies vulnerabilities.

PM for security products at GitHub here.

Totally agree. The good news is that I think GitHub is in a position to fix this - expect progress in the next 12 months.

One change that's needed here is data for each vulnerability on whether it's ever relevant in development. The advisory database that powers GitHub's security alerts (and npm audit, and NuGet audit) now has a dedicated curation team and is ready to curate more ecosystems and more information.

That data then needs to be hooked up with GitHub's security alerting logic. That shouldn't be too difficult - we already detect whether a dependency is used in development or production, and the team here is growing.

Finally, for this to work we'd need a new UI concept for vulnerabilities you aren't affected by. We're already working on functionality with a very similar requirement (one that tells you whether you're using the vulnerable function within a dependency).

I can't make promises, but I can say that GitHub has an increasing amount of energy. Expect progress :-)

PM for security products at GitHub here (and one of the original authors of Dependabot).

Sorry to hear that. I wouldn't expect us to be telling you about 1-5 security issues a day - do you maybe have (non-security) version updates enabled? If so and they feel like spam to you I'd recommend turning them off. (I wish I had a better suggestion, but until Dependabot supports grouped updates it sounds like it just isn't right for you.)

Dependabot doesn't support grouped updates yet but we hear the feedback and the team wants to work on them. Most of the investment in Dependabot recently gone towards improving our infrastructure and improving the experience for security updates. The team is still relatively small (it's 7 people), and supporting a service like this at GitHub scale is hard, but we're keen to keep improving.

HN is great, can I chime in with some feedback too, for NuGet I’m seeing Dependabot open separate PRs for the same dependency that’s out of date in the one repo that has multiple projects that relate to each other (they reference one another in csproj files).

They thus also need to be updated together, but I wish one PR was opened to update the dependencies in all projects at once, instead of multiple PRs I to merge.

PS: I think you should highlight dependabot updates on the Security tab in GitHub repo, I thought it was on before (but was actually just the security notices) because dependabot itself is hidden away in Insights -> Dependency graph -> Dependabot which was a bit surprising.

Feature requests always welcome! I'll pass it on to the Dependabot team.

Thank you

Let me know if https://github.com/github/roadmap/issues/148 is in the right spirit of what you're aiming for.

I think that what will get implemented will be a bit broader (covering version updates and security updates) and will allow for grouping by manifest, language, vulnerability, and potentially arbitrary labeling in the config file.

Yep, that looks good to me, thanks!

For NuGet, consider using centralized package references: https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-...

You get one file defining the set of all packages used in your repo (or some subset of your repo, etc.) and Dependabot will update this file directly. Individual projects can choose to use a package but won't specify the version.

It requires that your projects are all in sync with package versions but (1) that sounds like what you want (2) it's usually the best thing.

Thank you, looks like it’s still on preview but glad it’s being worked on.

It would be wonderful if there were bulk editing of Dependabot settings across multiple repos. The weakest part of GitHub is the lack of bulk administrative edits (repo settings, not code changes) across multiple repos.

This affects group (team) permissions as well - having to add a new team to multiple repos with specific permissions is a manual slog and hard to audit. Even the expanded permission categories are frequently inadequate - there's no CI focused "Let these users write to the repo and change hooks" permission level - all CI systems that self-manage hooks need full Admin rights.

The other code hosting I actively use is Gerrit, and the ability to hierarchically control what groups have specific permissions on multiple repos is far simpler and easier to use, from an admin perspective.

At a previous employer, we used Terraform to manage GitHub teams/repos/permissions.

It worked ... okay. Might be worth a try for you?

Can you expand a bit more on how you achieved this? It sounds interesting.

There's a GitHub provider for Terraform [1]. I tried it out on a GitHub Enterprise instance last year and found it a little awkward because they didn't have support for the latest GH APIs and the version before was missing a bunch of features. When I looked recently, they seemed to have got past that.

The principle is broadly the same as other Terraform workflows: import existing config and update it when you need to. You can manage repo permissions, people and teams, all centrally and in code. Super useful for tracking changes (the GH audit logs are a bit lacking in my experience) and identifying discrepancies between the expected and actual configurations.

[1] https://registry.terraform.io/providers/integrations/github/...

We did something similar to this at an org I worked at. We'd pull users from specific groups from our identity provider (Okta) and use that to provision corporate accounts to a Github Enterprise instance.

One big downside to this approach is Github teams often don't line up 1-to-1 with an organization's reporting structure, so you don't really ever get away from needing to manually manage user-team memberships, but it does remove the need to onboard/offboard users manually.

This is why we use GitLab over GitHub.

Puppet and Ansible are great for this type of things. If you want a new repo for your group, you make a pull request for our management repo.

There are a bunch of access rights, hooks and tokens that need to be set up in a certain way depending on the type of repo, and Ansible regularly makes sure no one deviates from the intended configuration.

Github (or Bitbucket or Gitlab) isn't different from any other software in this regard.

Does GitHub have "system level" webhooks like gitlab? That's what I've done previously, so when anyone creates a repo, it's configured as the org expects.

This page[1] seems to mention organization level hooks that receive repository created events, if making your team edit a terraform/puppet/ansible/chef repo to simply create a repo seems overkill to you

[1]: https://docs.github.com/en/developers/webhooks-and-events/we...

I'd love it if there was more org-level functionality in general, for both content and configuration.

> PM for security products

Please, please, _please_, add organisation scope to github action tokens (GITHUB_TOKEN in actions). It is painful that we have to create a PAT to access packages from within an organisation.

https://github.community/t/github-token-cannot-access-privat...

I'll make sure the Actions PMs see this. I _think_ I heard from a colleague that there's progress here.

> The team is still relatively small (it's 7 people), and supporting a service like this at GitHub scale is hard, but we're keen to keep improving.

While I empathize with your team regarding their apparent workload, that's only because they (probably) don't control hiring. GitHub isn't a struggling startup; it's not a non-profit; it's a mature, sizeable organization whose owner has over a trillion dollars in assets. If the team is too small to do a job, it's not because you lack personnel, it's GitHub's choice not to do it. You're GitHub, owned by Microsoft; GitHub doesn't care about these issues enough to staff them.

EDIT: Removed something provocative.

The Mythical Man-Month is probably older than you.

That book is an incredible read for anyone, even just the first couple chapters, but the central thesis is "you probably cannot pay any amount of money to make an over-due software development (or many other kinds of development) project be an on-time project", and not "there is no point in growing a team to solve bigger problems faster". You can't hire another orchestra to help play the same song faster, but you can hire another orchestra for more music over a longer time.

Yes, I thought of that, and it's a relevant point IMHO. But while hiring 10,000 people off the street won't improve things, GitHub hardly has its hands tied; they know how to get things done if they want to. We're talking about adding some basic features. Somewhere in GitHub and Microsoft, there probably are some additional highly productive developers who could be assigned to the project.

A must: allow Dependabot to merge PRs automatically. The reason to refuse this is frankly ridiculous [1]. For JS projects Dependabot is little less than a spammer, and the ability to automatically merge minor/patch versions is a must. Having to rely on third-party actions to do this is probably a larger security risk than letting people decide what PRs they want to automerge.

[1] https://github.com/dependabot/feedback/issues/954#issuecomme... and the original issue: https://github.com/dependabot/dependabot-core/issues/1973

And why it doesn't fly: https://github.com/dependabot/dependabot-core/issues/1973#is...

While we are on the feedback train, please consider making the Docker dependabot action more intelligent.

Right now, I have a monorepo with multiple Dockerfiles in subdirectories.

I end up having to add an entire docker block scoped to each directory that has a Dockerfile which is painful. Ideally, I would have a way to specify using */Dockerfile as a way to scan for Docker dependencies.

Also, multi-stage Dockerfile configs seem to confuse Dependabot where it'll submit a pull request to update the "AS build" but not the image to be used for packaging...

I really like the promise/premise of Dependabot, just want it to be easier to use (and edit across multiple repos).

> I end up having to add an entire docker block scoped to each directory that has a Dockerfile which is painful. Ideally, I would have a way to specify using */Dockerfile as a way to scan for Docker dependencies.

Glob syntax in `directory` would be sufficient, yes?

It's unclear if you are saying if that already works but yes that would be sufficient. =D

It doesn't, and was meant as a question

In that case, yes, glob syntax would work well.

Thanks for all the hard work, greysteil! Long-time Dependabot user here and a big fan (currently in the throes of updating many repos to GitHub-native Dependabot).

I've reported all this in the past, but since we're giving feedback in this thread, some big pain points for us are:

1. The lack of a way to view all Dependabot security alerts/security PRs (across repos) in one place. With Old Dependabot, we could do a GitHub Issues search like `is:open is:pr label:security org:<our-org>` and see them all, and actually built quite a lot of automation around PRs with the `security` label. But New Dependabot has no way to configure security PRs to get the `security` label, so it's easy to miss vulnerabilities when you have two dozen repos, and our automation no longer works. :/

2. Dependabot reporting `No security update is needed as <dependency> is no longer vulnerable` when in fact there are multiple versions of the dependency, including vulnerable ones, in our yarn.lock file. (Webpack has a lot of transitive dependencies!)

3. Dependabot can't update a vulnerable version of, say, rails, because rails requires the same version of activerecord, even though these are essentially the same project. (I get the sense this is being worked on as "grouped dependency updates"?)

P.S. Just wanted to clarify: feedback is love. I wouldn't be writing this if we didn't find Dependabot valuable (Old Dependabot was actually the reason we moved to GitHub years ago!).

Thanks for the feedback Jacob, and for all the support over the years. I'm going to pass those three on to the Dependabot team, who are best placed to think through solutions.

(FWIW, I'm sure we can think of something to fix (1) which balances security and usability - it just needs some thinking. (2) sounds like a bug to me - I'm sure the team will want to look at that. And (3) is a tricky case where Dependabot's Ruby logic needs to be able to unlock a dependency's "parent" or peers in order to make security fix (e.g., rails in the case where the vulnerability is in activerecord). Entirely possible but hard!)

It seems to primarily be a frontend thing. I get tons and tons of them, it's really annoying.

I also get a lot of spam from dependabot because of prereleases, I wish they would fix this bug: https://github.com/dependabot/dependabot-core/issues/2547

As others have pointed out, you can opt for daily / weekly or monthly updates, I'll stick to monthly until they fix this bug.

Hmmm, it's been a long time since I worked on Dependabot Core, but I think I can fix that one - we just need to special case `native-mt` as a version type (as opposed to applying standard Maven version comparison rules on it). I'll try and get a PR in today and will tag you.

Update: https://github.com/dependabot/dependabot-core/pull/4077

I usually ignore those and bundle a bunch in a single PR.

The worst I found was another team which merges in depandabot PR's one by one. Their git history isn't pleasant to look through :(

The git log command has a --perl-regexp flag [0], which allows you to filter commits that match a Perl-style regular expression. I guess in that situation you could specify a regular expression that excludes dependabot in --author.

If typing --perl-regexp gets old, you can always alias it or add a repo/global config option via grep.patternType [1]. Or you can create an alias for adding the less-powerful --invert-grep [2].

[0] https://git-scm.com/docs/git-log/2.32.0#Documentation/git-lo...

[1] https://git-scm.com/docs/git-config/2.32.0#Documentation/git...

[2] https://git-scm.com/docs/git-log/2.32.0#Documentation/git-lo...

Yup, my PR review requested page is near useless because of all of the dependabot PRs.

A setting I turned on to make it more manageable is to ignore patch updates and have dependabot only make PRs for major & minor version bumps.

it was easy: block the user.

then it got harder: create one branch that blocked the dependabot branch.

then it got harder again: create 00 00 00 - FF FF FF branches (let it run with parallelism and over night).

github support is not helpful on this.

Not recording exact dependency versions seems like the right way to stop those PRs from happening.

Dependabot PM following up a few days late

> Is there an option to tell dependabot "make one PR per week at most ...

You can set the `open-pull-requests-limit: 1` (https://docs.github.com/en/code-security/supply-chain-securi...) and the `schedule.interval: weekly` to limit the number of created PRs to one per week

> ... and bundle your changes"?

We've referred to this feature as "grouped updates" and it's tracked on the roadmap: https://github.com/github/roadmap/issues/148

Potentially using `allow: direct` (https://docs.github.com/en/code-security/supply-chain-securi...) to ignore the random sub dependencies, or ignoring minor versions (https://docs.github.com/en/code-security/supply-chain-securi...) of some/all dependencies might help reduce that noise.

Are the github's authors aware of Go's modules Vulnerability db?

https://golang.org/design/draft-vulndb

We are, and we're talking to them. In an ideal world we would have loved to have merged the databases.

I expect the two databases will move in sync. The GitHub advisory database is licensed as CC BY 4.0 (i.e., attribution only), so we actively encourage others, including the Go Vulnerability Database, to pull from it.

https://xkcd.com/927/

Can you.... not?

I was also wondering about this — I feel like it was just a couple of weeks ago that I read about the Go team's new unified vulnerability schema/Open Source Vulnerabilities (OSV) database

Definitely a step in the right direction on both ends, though!

https://osv.dev/list

https://security.googleblog.com/2021/06/announcing-unified-v...

https://opensource.googleblog.com/2021/02/launching-osv-bett...

I got few notifications yesterday for our golang projects. I was pleased to see less noisy alerts as compared to React based UI we have. It is terrible to say the least for npm projects.

Would love to know if dependabot creates noisy alerts for other languages too.

Had the same issue for Diem a while ago (I don’t work there anymore) and we ended up looking into software to manage dependencies in Rust. Check out https://github.com/diem/whackadep

meh. well that disappointing, its basically dependabot / graph and advisories.

Well maybe it’s meh if you already had it, but as a Go shop this is the first time we’ve been able to access these features so now I’m in party mode!

Yeah same for me. Seen it being used for npm and friends and been wanting some for Go too, for a long time now! I love graphs and dependency monitoring!

Well, yeah, the title says they are "bringing supply chain security features to Go" - so Go gets the same features other languages already have, no new exciting stuff to be expected here...

Reply on news.ycombinator.com