Show HN: WAPM – Package Manager for WebAssembly
wapm.io
wapm.io
I've grown uncomfortable with NPM being operated by NPM Inc instead of The Node.js Foundation, but it's a hard thing to change once it's established. We should hesitate to support the establishment of another community package manager by a for-profit company.
Here are some of the things that we are doing in order to resolve (or minimize) the concerns:
* Completely open API to retrieve the registry data (it's a work in progress, but our GraphQL API is open to everyone to use: https://registry.wapm.io/graphql )
* De-centralized hosting of the packages (news to come soon!)
Do you have ideas on how you will make the governance of the registry open? Who will make important decisions around policies, and how will they be made? I personally find this to be the core reason I find it hard to use startup-run package registries.
However, I would love to follow up with you personally to make sure we make the right choices from the start. Can you write me an email to syrus@wasmer.io so we continue the conversation there?
It is not obvious from your website, so if your success is tied to your open source solutions - as you say - how do you actually make money?
Is the index also going to be changeable? Will I be able to completely avoid your servers if I wanted to?
Given that "integration in package managers" is a big point in the IPFS 2019 Roadmap[0], I'm going to guess those two are related.
I really don’t think a private entity should be running a registry. Ruby is just about the only example of a registry handled correctly - via the community and non-profits.
It's more important to prevent centralization by demanding that there be no authority other than the end user, and by building tools to enforce those expectations, than to have a "holy grail" of dependency management be controlled by a single corporate interest.
I'd rather WAPM be built with their registry as a default option, but designed to be completely agnostic regarding registries or repositories. I'd like to use it and never have to touch their servers if I don't want to.
We’re not talking about good vs perfect. We’re talking about «a big ball of hairy non-compatible dependencies» vs «an upgradeable declarative definition of dependencies».
> I'd rather WAPM be built with their registry as a default option, but designed to be completely agnostic regarding registries or repositories.
That’s how npm works, yet you don‘t see git/vcs dependencies very often. That’s because they don’t work.
Ie, in Go it works just fine but it is more than just plain Git, there is a dependency file. Semver works with the repo itself based on repo tags.
I think you're not arguing against what the spirit of OPs post meant, aka the "like Go" part. Like Go does work, Go is using it and it includes a lock file just like you mentioned.
There are definitely downsides to using repos as dependency resolution hosts, but none of that in my view is what you mentioned. Lockfiles in repos are not complex or unsolvable, I'm not sure why you pit them as such - again, look at Go.
If you want to talk about why repos shouldn't be used, imo, talk about the volatility of them. A de/centralized host specially targeted at distributing source seems to have less volatility in existence when compared to git repos. It's rare to have packed disappear from Cargo (Rust's package manager), but I've had it happen multiple times in Go.
Example indexing service: https://godoc.org/
Example package: https://godoc.org/go.uber.org/zap
Making a new package manager is very expensive in terms of developer times - developers have to learn yet another new thing to use your platform. Can you justify this?
Edit: For example, as matthewbauer commented below, Nix can support WebAssembly as a compilation target: https://github.com/NixOS/nixpkgs/pull/56555
I think a far more interesting question is, does the package repository support falling back to e.g. curl. That would allow those who don't want another package manager to still use the registry.
[0] http://appfs.rkeene.org/ [1] https://github.com/crossmeta/cxfuse
And anyway, wasmer itself doesn't even support Windows.
> wasmer itself doesn't even support Windows
First, it will: https://github.com/wasmerio/wasmer/issues/51#issuecomment-44...
Second, I write code that doesn't run on my terminal all the time, but I need the packages I'm using to be installed and versioned correctly.
That has been the argument for the last several dozen new package managers.
I think that the cost in complexity for one Uber package manager is greater than the waste in duplication from many smaller, more focused ones.
How does the dependency model work? Can I safely install parallel streams of software? Does it support unprivileged installation? Is the installation stateless (no scriptlets/lifecycle scripts)?
I feel like the reason `npm`/the node.js ecosystem has these is because... as soon as you want to do anything serious, you need them.
How else do you build native modules across different platforms, run initial post-installation configuration scripts, etc.?
Scripts aren’t truly platform independent. You’ll branching logic for different platforms. Why not capture that with different declarative structures for each platform?
Running scripts as part of software install is something we, as an industry, need to solve.
It is my understanding that wasmer is trying to create a new ecosystem that doesn't rely on native modules.
> run initial post-installation configuration scripts
It's been my experience, as a long-time Linux user, that this is actually a bad thing. Stateless systems are far easier to work with -- a package manager can be far faster and simpler if it just extracts an archive. I can't think of a single case where post-installation configuration scripts couldn't be replaced by something simpler, using the filesystem.
Currently dependencies are keyed by namespace, name and version. There is a global namespace that has restricted access. WAPM does not do any dynamic linking of WebAssembly and currently only resolves dependencies one node deep. This will likely change in the future as the story on WebAssembly libraries and dynamic linking becomes more concrete.
> Can I safely install parallel streams of software?
WAPM operates synchronously at the moment, but there is no reason why WAPM should not be able to install dependencies concurrently.
> Does it support unprivileged installation? WAPM installs packages into a "wapm_packages" directory in the current directory. WAPM will probably support global installs in the future. WASI enables a "capability-oriented system", but this is a concern of wasm runtimes, and not WAPM. WAPM only manages wasm binaries.
> Is the installation stateless (no scriptlets/lifecycle scripts)?
WAPM installs with a single command and there are no lifecycle scripts. Ideally, one would install their wasm packages with wapm-cli and not require any intervention from other tools. It was designed to be unobtrusive.
I'm sorry, I didn't word that well. In Fedora, there is a feature called Modularity that enables you to switch between different release streams (for instance, Node.js LTS or current). You can do so with `dnf module install nodejs:11` or `dnf module install nodejs:10`. Since `dnf` is installing into a single global space, though, it isn't possible to install both nodejs:11 and nodejs:10, and the Fedora project recommends containers as a solution here. Could WAPM support this use case, without requiring one to buy into the entire container tooling ecosystem?
> no lifecycle scripts
Yes! This could enable a declarative OCI image build tool, which could be used to bridge the gap between Wasmer and the Docker/Kubernetes stack.
Reproducible builds are definitely something great, but they are quite tricky. However we are very open to hear more thoughts on how to do it!
One fairly simple thing you can do to improve this would be to include build scripts as part of the package, and allow people to run those build scripts through wapm. The exact versions of any involved tools will need to be recorded too, but since there arent too many ways to generate wasm blobs yet, this shouldnt be too out there.
Eventually it would be great if everyone had fully reproducible build formulae (kinda like https://github.com/polydawn/repeatr).
IMO any open source centric repository should have developers upload the source code instead of binaries so that the repository can compile it themselves to give binaries or source code to users. At the start they could use docker where the docker file is part of the uploaded artifacts, and later they could use WASI based toolchains directly, e.g. clang compiled to wasm or rustc compiled to wasm.
This "developers upload binary artifacts, source code is an afterthought" idea of npm rubs me wrongly.
You can still support proprietary software by just uploading the binaries as source (and maybe doing some build-process to adapt it to the packaging format)
Debian has developed an amazing set of standards for software repositories which I think should be applied universally to all package management ecosystems that want to have open source at their core.
E.g. they require that the source code has to be actual source code, aka "preferred form of modification" and not something minified. They also have separate repositories for proprietary and DFSG-free software, allowing users to choose whether they want to use proprietary software or not. Also, they run their builds without internet access, so the source code (as well as the dependencies tracked by the system) is actually everything needed.
There is a link on how to install from source, pointing to github, but the link is dead and the organization has no repositories...
At the moment, there is not enough momentum behind individual web assembly packages, as in "many people wanting to include the same wasm binaries". If at some point there are runtimes for like C#, Java, Python etc which people agree on, they might try and use a CDN approach, such that users visiting multiple websites don't need to download everything at every site.
It feels like this is more of a wasm "app store".
I don't think WAPM has any particular support for this (yet?), though.
Would love to hear your thoughts!
1. Package managers are very hard! The same problems wrt dependency resolution pop up again and again. Whatever you do, I recommend uhm, copying whatever Yehuda et al did for Ruby, Rust, Yarn, i.e. bundler/cargo/yarn
There's lots of prior art out there, and a few days of reading can save you months of terrible effort! Here's one example but there's loads more out there https://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn...
2. Namely, I don't see any lock files! I didn't look very hard. But get yourself some lockfiles. I also can't stress this enough.
3. Packaging signing really needs to be baked in from the get go, re: https://news.ycombinator.com/item?id=19733085).
Good luck!
Instead, people here are rightly cynical. I do not want another private company managing something as fundamental as a package registry.
I want them to answer my question about how they are actually going to fund themselves. Especially with the dubious decision to pay for office space in San Francisco.
I'd fund it through VC investment. It's a market/mind share play aka hosting a community.
1) How are you going to earn a profit? Like what is your revenue stream going to be, etc? Not funding.
2) Why do you need an office in downtown San Francisco? Why not work remotely?
btw it has "telemetry" enabled by default.
edit: thanks for the link, didn't find the source previously
We are also planning to open-source the registry as well :)
So I initially thought it was all closed source. Maybe add more GitHub repo links, so people can find the right stuff? :)
Although I guess the whole point, as far as the end user is concerned, is it doesn't matter what language it was written in. But it's still very relevant to people using them as libraries.
In case anyone is interested in any of that discussion.
Seems reasonable, I guess, though the power of wasm is that it can be run without install, like a webpage. We'll see if this catches on.