I think the point about most other languages having package systems that exist independent of the compiler is a great analogy. It really is a bit funky when you consider how tied to Node/V8 all npm modules are. And as the OP said, Node has a been a tremendously powerful, positive force in the world, but if JS really is the cat's meow, perhaps it's time for a real, framework-agnostic library manager.
Not all npm modules reference Node libraries, and most that do use them for features that are unavailable otherwise (file reading, http connections, etc...). But other than self imposed module requirements, I don't think npm actually has anything intrinsically tying it to node. You can run npm without node, and if the library functions were emulated in a different engine, I think it would work mostly without a hitch. It's just that nobody has done that yet.