feross

It's so fulfilling to see WebTorrent still popping up on Hacker News after all these years. I started the project in 2013 and devoted most of my 20s to working on it, ultimately becoming a full-time open source maintainer. I started WebTorrent with the goal of extending the BitTorrent protocol to become more web-friendly, allowing any browser to become a peer in the torrent network. Within less than a year of starting the project, I got WebTorrent fully working (see https://news.ycombinator.com/item?id=8317441). And it worked _well_, beating many native torrent apps in terms of raw download speed and the ability to stream videos within seconds of adding a torrent.

WebTorrent never got as much attention as the cryptocurrency projects selling tokens throughout the mid-2010s, even though WebTorrent _actually worked_, and it had more users than almost all of them :) I was never tempted to add a cryptotoken to WebTorrent, despite many well-meaning friends telling me to do it and cash in. Nonetheless, WebTorrent served as an accessible on-ramp to the world of decentralized tech, along with other projects like Dat (https://dat-ecosystem.org/) and Secure Scuttlebutt (https://scuttlebutt.nz/), playing a role in getting people excited about decentralization.

But WebTorrent is more than a protocol extension to BitTorrent. We also built a popular desktop torrent client, WebTorrent Desktop (https://webtorrent.io/desktop/), which supports powerful features like instant video streaming.

We also built a `webtorrent` JavaScript package (https://socket.dev/npm/package/webtorrent) which implements the full BitTorrent/WebTorrent protocol in JavaScript. This implementation uses TCP, UDP, and/or WebRTC for peer-to-peer transport in any environment – whether Node.js (TCP/UDP), Electron (TCP/UDP/WebRTC), or the web browser (WebRTC). In the browser, the `webtorrent` package uses WebRTC which doesn’t require a browser plugin, extension, or any kind of installation to work. If you’re building a website and want to fetch files from a torrent, you can use `webtorrent` to do that directly client-side, in a decentralized manner. The WebTorrent Workshop (https://webtorrent.github.io/workshop/) is helpful for getting started and teaches you how to download and stream a torrent into an HTML page in just 10 lines of code.

Now that WebTorrent is fully supported in nearly all the most popular torrent clients, including uTorrent, dare I say that we succeeded?

Not only that, but we helped the JavaScript ecosystem a ton by writing hundreds of npm packages including buffer (https://github.com/feross/buffer), simple-peer (https://github.com/feross/simple-peer), and StandardJS (https://standardjs.com/).

It's been a long and winding journey, but I'm glad to have played a role in making WebTorrent happen. Huge shoutouts to all the open source contributors to WebTorrent over the years, but especially Diego R Baquero and Alex Morais who were critical to WebTorrent's success.

If you're curious what I'm up to now... I'm building Socket (https://socket.dev) with an awesome team of open source folks. And there's actually a WebTorrent connection, too! Before Socket, we built an end-to-end encrypted file transfer app, Wormhole (https://wormhole.app), using WebTorrent under-the-hood (Show HN thread: https://news.ycombinator.com/item?id=26666142). Like Firefox Send before it, security was a primary goal of Wormhole (see security details here: https://wormhole.app/security). But one area where we felt we could improve the security of Wormhole was in how we audited our open source dependencies.

Like most teams building apps with JavaScript, we had a large `node_modules` folder filled with lots of constantly-updating third-party code. The risk of a software supply chain attack was huge, especially with 30% of Wormhole visitors coming from China. As most teams do, we enforced code review for our first-party code; but as most teams do, we pulled in third-party dependencies and dependency updates from npm without even glancing at the code. It's too much work to read every line of code of all dependencies. But the status quo would leave our users open to supply chain attack and we wanted to do better for our users. We looked around for a solution to detect signs of attack and to analyze the risk of various open source packages, but none existed.

So we built Socket to help developers ship faster and spend less time on security busywork by helping them safely find, audit, and manage OSS. By analyzing the full picture – from maintainers and how they behave, to open-source codebases and how they evolve – we help developers and security teams to identify risk from malware, hidden code, typo-squatting, misleading packages, and more.

Disclosure: I built this

Looking at package diffs is super important because of the rise of "protestware". For example, a maintainer of the event-source-polyfill package recently added code which redirects website visitors located in Eastern European timezones to a change.org petition page. This means that real users are being navigated to this random URL in production.

See the attack code here: https://socket.dev/npm/package/event-source-polyfill/diff/1....

It’s very unlikely that users of event-source-polyfill are aware that this hidden behavior has been added to the package. And yet, the package remains available on npm many months after it was initially published. We think that supply chain security tools like Socket have an important role to play in warning npm users when unwanted ‘gray area’ code is added to packages they use.

Socket watches for changes to “package manifest” files such as package.json, package-lock.json, and yarn.lock. Whenever a new dependency is added in a pull request, Socket analyzes the package's behavior and leaves a comment if it is a security risk.

You can see some real-world examples here: https://socket.dev/blog/socket-for-github-1.0

Is this standard practice in most states?

I'm not too worried about the npm condition. My reading of it is that it's intended to prohibit using security data generated by npm itself. When talking about "data about the security of Packages" they give the examples of "vulnerability reports, audit status reports, and supplementary security documentation". We don't use any of that stuff.

Socket is built by a team of open source maintainers (everyone on the team is a maintainer!). We're motivated to defend the open source ecosystem from supply chain attacks.

We'll be watching this post, so feel free to ask us questions!

- You can’t trust the package registry because of security flaws in the registry itself (as seen here as well as in NPM just a few months ago[1]). The CDN can be hacked, or an insider can attack the infrastructure.

- You can’t rely on code signing since a maintainer can go rogue and sabotage a package at any time (as happened with colors.js and faker NPM packages in January[2]) or a new maintainer could be added to the project and possess a valid signing key but turn out to be a bad actor (as happened with the event-stream NPM package in 2018[3]).

- You can’t audit every line of code in every dependency because the cost in terms of time and expertise is prohibitive to all but the biggest organizations (e.g. Google) and the most security sensitive applications (e.g. certain crypto projects and financial applications).

The two solutions I’m most excited about are (1) auditing package behavior with static analysis to detect when package behavior changes (e.g. new network connections, filesystem accesses, install scripts) like we do at https://socket.dev (disclosure: I am the founder) or (2) package sandboxing of which Lavamoat is the best example (however the performance impact is too high to apply this to every package in an app at present, and it also requires maintaining a policy configuration for each package).

[1]: https://www.theregister.com/2021/11/16/github_npm_flaw/

[2]: https://www.theregister.com/2022/01/10/npm_fakerjs_colorsjs/

[3]: https://www.theregister.com/2018/11/26/npm_repo_bitcoin_stea...

Yes. You are responsible for the code that you ship to production, whether you wrote it or it comes from open source. Fundamentally, there's no other way to solve this problem in a dynamic language like JavaScript.

You can use tools like https://socket.dev (disclosure: I'm the founder) to automate finding the dependency updates that are particularly suspicious/risky, i.e. they contain the tell-tale signs of a supply chain attack, including the introduction of install scripts, obfuscated code, high entropy strings, or usage of privileged APIs such as shell, filesystem, eval(), and environment variables.

In the far future, I have hope that efforts like ES Realms and LavaMoat will give us per-package permissions and true isolation, but at the moment they're not practical/performant enough to use. Even with code signing (an often suggested approach), you still have the problem of maintainers going rogue, or maintainers adding new malicious maintainers to formerly safe dependencies.

tldr; you need to 'review' every patch, though it doesn't necessarily need to be a human review for every patch.

A few other ways that attackers get write access to packages that were not covered:

1. Maintainer gives access to a bad actor. The most famous case happened in November 2018 when Dominic Tarr gave access to the package `event-stream` to a new maintainer who turned out to be an attacker. This bad actor offered to help maintain the package – something not uncommon in open source. At the time of compromise, event-stream had 1.5 million weekly downloads [1]

2. Maintainer goes rogue (i.e. becomes an attacker). The most famous case happened in January 2022 when a maintainer named Marak 'went rogue' and sabotaged his own packages `color` and `faker` which together received over 20 million weekly downloads [2]

I believe that the way forward is to assume all open source packages may be malicious. At socket.dev, we use "deep package inspection" to characterize the behavior of an open source package. By actually analyzing the package code, it's possible to detect when packages use security-relevant platform capabilities, such as the network, filesystem, or shell.

For instance, to detect if a package uses the network, socket.dev looks at whether fetch(), or Node's net, dgram, dns, http or https modules are used within the package or any of its dependencies. We can detect the tell-tale signs of a supply chain attack, including the introduction of install scripts, obfuscated code, high entropy strings, or usage of privileged APIs such as shell, filesystem, eval(), and environment variables. [3]

[1]: https://blog.npmjs.org/post/180565383195/details-about-the-e...

[2]: https://www.bleepingcomputer.com/news/security/dev-corrupts-...

[3]: https://socket.dev

From recent research:

> We found 93.9% (3,412) of malicious packages had at least one install scripts, indicating that malicious attackers use install scripts frequently [1]

When malware authors adapt and start doing fancy dynamic stuff, we might not be able to figure out exactly what they're doing, but we can detect the usage of obfuscated code, dynamic requires, and other signals of compromise.

[1]: https://arxiv.org/pdf/2112.10165.pdf

> It would be even better if something like this could be integrated directly into the package management tool itself

We're planning to build this. However right now, the primary way to consume Socket.dev data is through our GitHub app (https://socket.dev/integrations).

We're working on improving our analysis and are close to shipping a big update at which point we'll increase the severity of these issues.

Is this the type of list you were looking for?

It's not a perfect metric and isn't meant to disparage any of the maintainers of these packages, but it's one factor in our Socket.dev risk analysis.

There's a few reasons that NPM sees more attacks than other ecosystems.

First, the scale of the JavaScript ecosystem. JavaScript is so much larger than every other ecosystem, so even a very small probability event (somebody introducing malware into a package) can happen surprisingly often given the scale of the ecosystem. Supply chain attacks are a problem in all open source ecosystems – not just JS – but they are a bit rarer and don't effect as many people so fewer people take note.

Second, npm was one of the first package managers to solve the classic "dependency hell" problem. In Python, if you have two dependencies, A and B, which both depend on different versions of C, say C@1.0.0 and C@2.0.0, respectively, then you're in trouble. You have an broken project. Python can only install one version of C. So now you're in dependency hell.

Npm on the other hand just installs both versions of C and it gives A the version that it wants, C@1.0.0. And it gives B the version that it wants, C@2.0.0. Both packages are happy - problem solved.

This caused Python maintainers to think twice before adding a new dependency lest they cause "dependency hell" for their users. Much better to just copy paste these 50 lines of code rather than adding a dependency. So there was an intrinsic sort of resistance – some pain is involved in adding new dependencies.

Npm maintainers had no such constraints. In a way, npm’s better developer experience led to the whole module ecosystem scaling "too well". Thus, you end up needing to trust more total maintainers, increasing the risk of supply chain attacks.

Disclosure: I started Socket (https://socket.dev) to help solve open source supply chain security. To learn more, see: https://news.ycombinator.com/item?id=30521913

For example, see the package `angular-calendar` which is a calendar/date picker web component. When you look it up on Socket.dev [1], you'll see that it actually uses:

- Install scripts

- Telemetry to track you

- Network access

- Shell access

- Environment variable access

- File system access

Which is waaay more capabilities than you'd expect. All of these capabilities turn out to be caused by a single dependency which implements telemetry to track the package usage a la Google Analytics.

[1]: https://socket.dev/npm/package/angular-calendar/issues/0.29....

I totally agree with the idea that we should assume all open source packages may be malicious. Socket.dev uses "deep package inspection" to characterize the behavior of an open source package. By actually analyzing the package code, Socket can detect when packages use security-relevant platform capabilities, such as the network, filesystem, or shell.

For instance, to detect if a package uses the network, Socket looks at whether fetch(), or Node's net, dgram, dns, http or https modules are used within the package or any of its dependencies.

This entails running static analysis (and soon, dynamic analysis) on a package – and all of its dependencies – to look for specific risk markers.

In this way, Socket can detect the tell-tale signs of a supply chain attack, including the introduction of install scripts, obfuscated code, high entropy strings, or usage of privileged APIs such as shell, filesystem, eval(), and environment variables.

We are taking an entirely new approach to one of the hardest problems in security in a stagnant part of the industry that has historically been obsessed with just reporting on known vulnerabilities.

Snyk and the entire security industry is obsessed with identifying known vulnerabilities. But they all miss the point. Looking for known vulnerabilities is reactive. Vulnerabilities take weeks or months to be discovered.

A malicious dependency can be updated, merged, and running in production in days or even hours. We need to assume all open source may be malicious and try to proactively detect indicators of compromised packages. A better approach is to detect when dependency updates introduce new usage of risky APIs such as network, shell, filesystem, and more.

And thanks for the WebTorrent love!

Snyk and the entire security industry is obsessed with identifying known vulnerabilities. But they all miss the point.

Looking for known vulnerabilities is reactive. Vulnerabilities take weeks or months to be discovered. A malicious dependency can be updated, merged, and running in production in days or even hours.

We need to assume all open source may be malicious and try to proactively detect indicators of compromised packages. A better approach is to detect when dependency updates introduce new usage of risky APIs such as network, shell, filesystem, and more.

Disclosure: I started Socket (https://socket.dev) to help solve open source supply chain security.

First, the scale of the JavaScript ecosystem. JavaScript is so much larger than every other ecosystem, so even a very small probability event (somebody introducing malware into a package) can happen surprisingly often given the scale of the ecosystem. Supply chain attacks are a problem in all open source ecosystems – not just JS – but they are a bit rarer and don't effect as many people so fewer people take note.

Second, npm was one of the first package managers to solve the classic "dependency hell" problem. In Python, if you have two dependencies, A and B, which both depend on different versions of C, say C@1.0.0 and C@2.0.0, respectively, then you're in trouble. You have an broken project. Python can only install one version of C. So now you're in dependency hell.

Npm on the other hand just installs both versions of C and it gives A the version that it wants, C@1.0.0. And it gives B the version that it wants, C@2.0.0. Both packages are happy - problem solved.

This caused Python maintainers to think twice before adding a new dependency lest they cause "dependency hell" for their users. Much better to just copy paste these 50 lines of code rather than adding a dependency. So there was an intrinsic sort of resistance – some pain is involved in adding new dependencies.

Npm maintainers had no such constraints. In a way, npm’s better developer experience led to the whole module ecosystem scaling "too well".

Disclosure: I started Socket (https://socket.dev) to help solve open source supply chain security. To learn more, see: https://news.ycombinator.com/item?id=30521913