Hacker News Reader

But what would be the point? Why would you load code over HTTPS if you're going to make it static anyway? If the code can't change dynamically, then you might as well put it next to the JS file on the filesystem.

I think this is a terrible idea that can only end in misery but I can understand why they don't add integrity validation by default.

HTTPS doesn't magically protect you from the content of the package changing.

They are different security mechanisms designed to defend against different threats.

One answers the question: Was this the correct server I got this package from?

The other answers the question: Does the package's hash match what I expect in my code?

As to why you would want this, I've worked on several projects that take advantage of some things that pulling dependencies only from a local repository can give you. Take the recent Log4J vuln as an example. Immediately on seeing that vulnerability, I disable the JAR file, causing builds to fail wherever that dependency exists until the updated one is added as a dependency and used. From experience having to do dependency analysis on an old toothy codebase, it's sometimes simpler and more accurate to just let everything break and fix the breaking spots.

I understand that HTTPS doesn't protect you, and that's the reason why you'd normally add a hash.

If your goal is to make the build fail for vulnerable applications, you can run a local repository with upstream proxying functionality like Nexus Repository and kill the package there. This won't let you kill production (but why would you want to kill it this way?) but it'll give you the same benefit without risking remote code execution.

If you understand HTTPS doesn't protect you, I'm unclear on your initial point. Any chance you can elaborate on precisely what you meant when you said:

    But what would be the point? Why would you load code over HTTPS if you're going to make it static anyway? If the code can't change dynamically, then you might as well put it next to the JS file on the filesystem.

    I think this is a terrible idea that can only end in misery but I can understand why they don't add integrity validation by default.

I provided an example of why you would want to still load it remotely (specific situations where it's beneficial) rather than simply copying the code into the source repo, which you have also echoed, though seemingly suggesting that the example means you can do away with integrity checks?

If my read on that is correct, then point then is to worry about the repository - wherever it lives - being poisoned, but still wanting some of the benefits of having a remote repository (such as centralized dependency management and control).

> I provided an example of why you would want to still load it remotely (specific situations where it's beneficial) rather than simply copying the code into the source repo

Doesn't Node cache imports, though? I think this approach would protect you if you were running a script on a timer (thus triggering a fresh load at a regular interval) but not if you were running an Express server (which would only verify the hash of the remote library at application startup).

It’s a feature that Deno has and that works on the web (ESM imports from URLs). People have use cases for it and they want it. That’s not to say it’s secure by default of course but the demand is there.

There's always demand for completely insecure features. That doesn't mean you should add them.

The code can update dynamically. You simply have to update the version number and verification hash on your end (same as you would in a package manager). This is still vastly easier than copy-pasting entire repositories into your codebase every time.

Right, but if you don't want the code to update unless you specifically update the verification hash and version number, why wouldn't you download and install it locally?

There's two separate questions:

A) is it ever good to have dependencies that can change underneath you? (arguably no)

B) if you don't want dependencies that change underneath you, why would you ever want to import directly from a URL at runtime?

----

> The code can update dynamically. You simply have to update the version number and verification hash on your end (same as you would in a package manager).

Code that I import through a package manager can update dynamically. I just need to update the version number and re-run `npm install`. Why would someone want to use a URL if the update process is the same as using a package manager?

I'm having a somewhat difficult time imagining a use-case for fetching dependencies at runtime that isn't related to not knowing until runtime what the dependencies are or what version is being fetched. Maybe for making installation smoother? But anything that requires compilation or is system-dependent isn't something you're going to want to re-fetch at runtime anyway. I suspect those dependencies will be slow to install.

And we're not dealing with a web situation where you might want to import from a CDN or another location that has better uptime. It sometimes makes sense not to serve assets yourself on the web just because it's expensive/difficult to serve them quickly. Node dependencies don't have that problem, pulling the dependency in as an install step will mean runtime performance is faster, not slower.

I get the argument that you shouldn't do dynamic imports that don't have resource integrity checks, but that seems like an argument against this feature in general -- you shouldn't really be doing runtime imports at all, because... why would you need them except for unpinned/unchecked dependencies?

The advantage of this approach is that you don't need a separate package management ecosystem, no central hosted registry, no submissions. A bit of code is hosted at a URL, and that's it. It uses the same ES standards as importing any another JS script from your filesystem.

Secondly, this doesn't mean all this code has to be dynamically loaded at runtime. You can still have equivalents of npm install which cache the linked resources locally on your server, perform integrity checks etc (this is exactly how Deno does it).

> The advantage of this approach is that you don't need a separate package management ecosystem, no central hosted registry, no submissions.

But npm has this already. You can already install from urls, you can even install from remote git urls. It's not even complicated, it's just `npm install git+ssh://user@hostname:git_repo.git#commit-ish`.

I get that npm isn't Node, but this feels like a lot of duplicated effort for something that is much more safely handled with the package manager that ships with every single default Node installation. It's not exclusive to npm either, Yarn supports this stuff too.

Are there really people building Node projects who are using a package manager that won't let them import from urls?

----

> Secondly, this doesn't mean all this code has to be dynamically loaded at runtime. You can still have equivalents of npm install which cache the linked resources locally on your server, perform integrity checks etc (this is exactly how Deno does it).

Unless Deno is crawling all of the source files in the project and doing this before the code is even run, aren't you still going to run into situations where you run your project and it hits an import it didn't see before, and it then gets dynamically imported at runtime?

On the web, ES imports are designed to be kind of inflexible and static so that you can't have those dynamic systems, it's all designed to make it harder to run into unexpected imports and easier to do static analysis of dependencies. And maybe Deno is the same, but if so then I don't see what the advantage is over doing that instead of putting all of your dependencies in a package.json and fetching them as part of an explicit step. The web made this stuff super static and inflexible to try and make dependencies more explicit, but an install step is kind of the end-goal of those restrictions, it's the most explicit way possible of importing dependencies.

And remember that npm lets you get your dependencies from anywhere, and they all get packaged in a predictable way, transparently on disk. You don't lose any flexibility in regards to where you get your packages from. At best, this system does the same thing, in which case why are we duplicating something that already exists? At worst, it caches resources only for future runs after it fetches them the first time, and it fetches them the first time at runtime, in which case I just don't understand why that would that be desirable.

Maybe conditional installs of packages so you can only fetch a subset of the potential dependencies you might need based on what is actually used during program usage? But this seems like a really error-prone, potentially dangerous way to achieve that.

> But npm has this already. You can already install from urls, you can even install from remote git urls

Yes, but npm can only install npm packages. This is a step towards reducing/removing the dependency on npm itself from the Node ecosystem. It is duplicated effort, sure, but that effort is spent in undoing extra complexity and getting to a much easier "hello, world" state for both app and library developers.

> Unless Deno is crawling all of the source files in the project and doing this before the code is even run

That's exactly what it is doing.

> Yes, but npm can only install npm packages.

I don't understand what you mean by this. What is an npm package?

Node already manages how `node_modules` get imported from disk, and it handles the package.json format; that stuff is already part of Node. The only thing that npm does is fetch the packages/dependencies, stick them on disk, and then do integrity checking with package-lock.json. And all of that is already somewhat optional. Even when you're using npm, there's no certification you need to go through, you don't need to use a proprietary format, you don't need to get anyone's permission to distribute your package, you don't need to sign anything. You need to write a bit of JSON and then zip your source into a tarball, and then anyone can use any Node-compatible package manager to import it and its dependencies.

npm isn't even a requirement for fetching packages or pinning resources, you can use something like Yarn as well (and arguably should, because Yarn has much better dependency pinning). You can even hand-write your own compressed packages that work with npm.

> It is duplicated effort

It is duplicated effort of functionality in NodeJS. And it's duplicated effort of good functionality, Node's `node_modules` system is decently good design. To the extent that we are getting rid of being able to use 3rd-party tools to fetch packages, we shouldn't do that. npm/Yarn have already proven that having multiple ways to import packages is beneficial, and being able to dictate one program that's in charge of how dependencies are checked for resource integrity and where they get stored on the disk is also good. We only got package-lock.json in npm after Yarn implemented it.

But I guess that's arguable and maybe someone thinks that we shouldn't have a package manager at all. I disagree, but sure, I'll concede that it would be reasonable for someone to think that.

> That's exactly what it is doing.

So for whatever reason, Node wants to handle fetching resources itself. Fine, we'll say there's a good reason for that. Having said that, when we look at the proposed syntax, why would you want your dependency declarations spread throughout your code rather than having all of your dependencies defined in one place? This sounds like a strict downgrade in terms of auditability, code complexity, etc.

You're saying that we're reimplementing things so we can undo extra complexity, but how is parsing out an import tree from source files simpler than reading a list of imports out of a package file? If the goal is to make a simpler system, I kind of like having a system where I can audit dependencies by hand rather than looking through source files and making a list of all of the imports.

Not to mention that this system also isn't flexible enough to handle dev-dependencies or global dependencies, so we're going to end up adding a bunch of complexity and extra syntax for that I suppose if we really want to get rid of npm/Yarn.

----

To play devil's advocate, I'll grant that now you can kind of imitate ES Imports without changing syntax. If I remember correctly, that was part of Deno's reasoning for using URL imports in code.

But again, why? ES Imports are a compromise due to the structure of the web and the fact that fetching packages at runtime on a website matters. Why hobble a native system with an import strategy that realistically wouldn't have been chosen for the web if any other option had been available?

If I'm understanding you correctly, this has built an install step that works the exact same way as a normal install step for languages with normal package managers, except it doesn't have strong verification guarantees, and it's harder to audit by hand, and (assuming I'm understanding the docs[0] correctly) no longer goes through an explicit install step and requires you to actually boot up the app once before it'll grab the dependencies.

Saying that Node wants to get rid of npm still doesn't answer the question of why someone would want what sounds like a normal install step to be spread out as if it's being imported dynamically. We're native now, we don't need to make the same compromises that web imports were forced to make. We can have an explicit install step with dependencies that are organized into a human-maintainable list. So it feels like we've gone full circle from "why would someone want dynamic imports if they're pinned" to "they wouldn't, and they're not dynamic imports". And I guess my followup question is, why would someone want statically defined imports that kind of pretend to be dynamic imports but actually aren't?

[0]: https://github.com/bmeck/node/blob/ceadb473e6d3a22f38b737108...

>But npm has this already. You can already install from urls

One use case is if you have a JS module that depends on another JS module, and you want to be able to run it without transpilation in both Node applications and in browser environments. NPM-installed modules wouldn't be available to the browser engine, regardless of how NPM installed them.

> If the code can't change dynamically, then you might as well put it next to the JS file on the filesystem.

Yeah, you might be on to something with that!

we could call it dpm “Deno Package Manager”

Exactly, this kind if import without a hash validation is a big no for security reasons (unless you 100% trust your import source). This feature exists on the browser side with the script element: https://developer.mozilla.org/en-US/docs/Web/Security/Subres...

> you 100% trust your import source

This is a perfect circumstance for the adage "trust but verify".

There's already syntax to implement it with imports through import assertions: https://v8.dev/features/import-assertions

It's rubygems all over again. I think only Nix and Go are doing dependencies right (cryptographically verified) these days.

One upside is that it's https-only, which does include authentication, provided that you trust the holder of the certificate for my_local_repo.tld (in your example) to serve you the right files.

Don't forget about PHP. Composer is one of the best package managers around.

Came here to say the same thing. In fairness, this isn't specifically Node's fault. I feel that a mistake was made early in the SPA/mobile craze a decade ago, where the web should have used subresource integrity (SRI) everywhere:

https://developer.mozilla.org/en-US/docs/Web/Security/Subres...

But instead went with less-secure Cross-Origin Resource Sharing (CORS):

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

These operate at different levels of abstraction. SRI is mathematically strong, whereas CORS requires a bunch of hoops and mostly just gets in the way. I could understand an argument for SRI+CORS, but using CORS alone sets off alarm bells in my head.

CORS and SRI are just completely different. CORS exists to allow requests that would be banned under the Same Origin Policy, most of which have dynamic responses and could not integrate with SRI at all.

Just for anyone curious, I know that SRI and CORS are different. SRI negates the need for CORS in some situations because we're saying that we only care about the file content, not where it came from.

For the web, we could have been sourcing all of our javascript files from a central CDN and used the hashes to prove that they weren't tampered with this whole time. Instead, we went down the rabbit hole of build systems and endless local hosting.

For this Node.js use case, they could have allowed remote includes, with the understanding that it's only secure when the hash is specified. Verifying only the location is like CORS, it's security theater based on trust instead of a mathematical level of confidence about the contents.

Good point about dynamic responses not being compatible with SRI though. I hadn't thought about that use case, but you might be onto something about its lack of adoption. It may be a red herring however, because to make SRI dynamic, we'd need some kind of trust process that tells us that a new version's hash is safe, more like a BitTorrent tracker, but the file would still be requested through ordinary SRI. Also it doesn't apply to what we're talking about here, because package versioning means that the files are always static anyway.

I agree - I'm not really sure what the parent poster is on about. CORS and SRI are orthogonal, and designed to solve entirely different problems.

CORS is primarily focused on removing csrf (cross site request forgery) vulnerabilities. It doesn't really care at all about what resources you pass around (or their integrity) it only cares about ensuring that the request is coming from a known and allowed origin.

> CORS is primarily focused on removing csrf (cross site request forgery) vulnerabilities

basically CORS gives you zero defense against csrf. (ok, not zero, but also not much)

This just isn't true. I don't consider request forgery to just be side-effect based (the classic example of a bank form post). I'm including any request that leaks data to a site based on cookies present only on the client (forged authenticated request).

SOP/CORS was the foundation that allowed CSRF tokens to even work. (No SOP/CORS? Just request the form with the csrf token first, parse with js, append to next request)

I guess you could say SOP is really the protecting force here, but CORS only exists because strict SOP broke a LOT of stuff. Enter trust zones in IE, and then CORS.

SameSite now mostly tackles this by fixing the root issue with the cookie jar.

cors never protected against csrf because you could just have a form that gets submitted via js. that worked for ALL post requests and it works in modern browsers as well and most csrf attackes used forms.

I’m also concerned about resource integrity, and that was my immediate concern initially when I heard of Deno. But it’s important to also recognize that it’s quite a bit more complex than a hash for the requested module. Because of course that hash has nothing to say about the integrity of transitive imports.

I’m willing to give Node the benefit of the doubt that this will remain experimental with appropriate warnings until resource integrity is addressed to some reasonable level of confidence. I’m also glad to see it being a first party concern, especially given that it’s historically been/still is a much higher risk factor as both:

- delegated to their ESM loader API, which is incredibly difficult to get right, despite the API itself being deceptively simple

- a hilariously insecure example in their docs for same