Hacker News Reader

Comment by stevebmark | Hacker News Reader

I've always thought that dependabot was busy-work, a waste of time. This article makes a good point that drives it home: Alarams that aren't real make all alarms useless. Dependabot is especially painful in non-typed languages (Python, Ruby, and especially Javascript) where "upgrading" a library can break things that there's no way to know until production.

Maybe the constant work, extra build time (and cash for all that), and risk of breaking production, is worth it for the 0.01% of the time there's a real vulnerability? It seems like a high price to pay though. When there are major software vulnerabilities (like log4j), the whole industry usually swarms around it, and the alarm has high value.

I just realized how much CircleCI probably loves Dependabot. I wonder what hit % their margins would take if we moved off it collectively as an industry.

View on HN

> When there are major software vulnerabilities (like log4j), the whole industry usually swarms around it, and the alarm has high value.

You're leaving me with the impression that you think we should only patch major software vulnerabilities. This I would disagree with. Minor vulnerabilities can be used, especially in groups, to do things we don't anticipate. It's not just about a single vulnerability but about how an attacker can leverage multiple different vulnerabilities together.

If you use vendoring, it's also worth considering that there's always some inherent security risk in upgrading dependencies. If an attacker takes control of a package somewhere in your dependency tree, you don't get compromised until you actually install a new version of that package. This risk can often outweigh the risk of very minor/dev-facing CVEs.

Shameless plug: This is what I’m building Socket.dev to solve.

Socket watches for changes to “package manifest” files such as package.json, package-lock.json, and yarn.lock. Whenever a new dependency is added in a pull request, Socket analyzes the package's behavior and leaves a comment if it is a security risk.

You can see some real-world examples here: https://socket.dev/blog/socket-for-github-1.0

We use Socket and my favorite feature is when you highlight new dependencies with a post-install hook. It’s not always a problem, but almost always a smell.

One feature request: please allow me to “suppress” warnings for a specific package+version combo. This is useful for activist libs that take a political stance - I know it happens, but often cannot remove them, and don’t want to continue flagging the same problem at every sec review.

I kind of feel like dependabot alerts should be treated like a coding convention error - that extra whitespace isnt actually causing a problem but we fix it right away.

Otherwise you have to start analyzing the alerts, and good luck with that. The low severity ones are marked critical and the scary ones are marked low. Suddenly you have 200 unfixed alerts and its impossible to know if somewhere in that haystack is an important one.

Impossible? The article we're commenting on describesa tool that does this.

> can break things that there's no way to know until production

I would argue that any production system should have enough tests that upgrading a dependency that breaks compatibility should cause failure of the test suite in some way

You can argue that but I've seen plenty of production systems relying on dynamically typed languages where breaking third-party dependency changes were indeed undetectable until a new version was deployed and particular features stopped working at runtime. It's certainly not unheard of for the same to occur with statically typed languages but at least there's a whole class of potential errors that are caught automatically by compilers/linkers with no extra work on behalf of the programmers. And even when errors do occur at runtime due to JIT compilation etc. they're more likely to have helpful information attached to them. Either way, all the automated tests in the world aren't likely to catch a problem that's related to incorrect library versions being loaded from some global cache that you have limited control over (having moved back from SaaS to on-prem based software recently I've been very quickly reminded of that reality!).

This is a very good argument for why Ruby should never be used for production systems, and neither should Javascript without using Typescript.

Why specifically Ruby here? How about python, php, etc? Also, many dynamic languages have static type checkers if you want to use them (i.e. Ruby has Sorbet).

IMO Dependabot is really dreadful at its job. Try Renovate - it's really brilliant, fast, flexible, supports properly binding PRs/MRs.

> where "upgrading" a library can break things that there's no way to know until production.

unless you have automated testing

Reply on news.ycombinator.com