Opensourcing the Universal Package Manager | Hacker News Reader

Opensourcing the Universal Package Manager

repl.it

163 pointsamasad40 comments

Repl.it founder here. Happy to answer questions. But also I'd like to point out that this is not the only "universal" project we're working on. See:

- Prybar: Universal REPL interface https://github.com/replit/prybar

- Polygott: Universal docker image for all languages (we support) https://github.com/replit/polygott

Between these three you'd be able to add your favorite language to Repl.it (patches welcome). We have a write up on how to do that using elisp as an example: https://repl.it/site/blog/elisp

As a regular user and promoter of REPL.it, I like it very much and therefore submitted two PR’s [1][2] after seeing a blog post about how to contribute to it [3].

Unfortunately, those PR’s have been lingering up there for the past 4 months without any meaningful feedback. I fully understand that you are a business and therefore there are priorities but it would be great to get any kind of feedback on them in order for me proceed with further actions one way or another.

Anyway, thank you for providing such a nice tool for free and open-sourcing it!

[1] https://github.com/replit/polygott/pull/28

[2] https://github.com/replit/prybar/pull/30

[3] https://repl.it/site/blog/elisp

Hey Salam, thanks for your PRs, definitely appreciated. I'll figure out what's the hold up -- I'm sure there is a reason -- and I'll make sure to get back to you

Wow, that's very cool.

I'm curious though, won't open sourcing all of these make it much easier for competitors to catch up to your head start in these areas? I wonder what your risk/reward thinking is wrt deciding what to open source.

(NOTE: I'm not associated with repl.it)

It might make it easier for competitors to catch up. It might also turn some who might have become competitors into collaborators.

Sure it well. We have a lot of copycats as well. But as Elon Musk likes to say: the only moat that matters is the continuous pace of innovation and the value of open-source outwieghs any contrived barrier to entry.

This is pretty cool! Lots of "configuration package managers" would benefit from a universal interface, and many of these boil down to "directory of git repositories at a specific commit". Some examples: many dotfiles repositories, pathogen (vim plugins), fisherman (fish shell; exrok/fisherman).

It would be nice to have a unified way to manage these, and even better if that unified way was through UPM, the same way I install new nodejs, python, whatever packages.

Exactly mirrors our thinking!

Our number one priority is that the programming experience on our site is flawless. We're playing hardmode though because we're dedicated to making the experience across languages the same. And that's why we create abstractions like this.

I'm hoping a project like this could at least start the conversation around standardization of interfaces.

This is awesome!

I’d love to work on an integration with https://wapm.io (WebAssembly) and https://crates.io/ (Rust) if you think that could be useful!

Yes please! Hmu if you'd like to chat about it.

How would integration with Rust (cargo/crates) even be possible?

Looking at how Go is so different in dependency management (Go modules intentionally broke from the config/lock file format)... I'm not sure it could be used there. I wonder if there are any other end cases that wouldn't work.

Crates would totally be possible. AFAICT it follows the exact workflow we're trying to standardize.

If they could take the output of a command to get the 'lockfile' type information, they should be able to get something working for Go as well. Presumably they have allowances for this sort of thing as to be 'Universal' would require that type of flexibility.

Ctrl-f "nix": zero results. Come on. What's the problem? Nix isn't my favorite language but this is just getting to be stupidly silly. Every three months one of these posts pop up, never going anywhere, never addressing the already existing nearly universal package manager that literally powers tons of workstations/servers/containers top-to-bottom.

Nixpkgs currently contains:

- the linux kernel (and supporting bits+userspace to build an entire bootable OS installer/disk image/PXE payload, etc)

- tens of thousands of Linux packages (many of which also work already on Darwin, where applicable)

- thousands of native {Go,Python,Haskell,Node} dependencies

If half of the effort that has gone into Linuxbrew and other efforts like this, went into nixpkgs, it would probably have even more packages and Windows support and we could stop inventing a new package manager to rule them all every 6 months. Insert XKCD joke here.

edit: -1, zero replies, years of being here and nothing changes. gotta love it. Maybe do more research next time instead of NIH?

edit2: I'll donate $500 to the charity of your choice if you can show me a "universal package manager" that contains more packages than nixpkgs (which has existed for ages at this point). I'll wait.

Yes Nix has been around for a while and is a very good package manager. But please don't be disrespectful to everybody who doesn't know about it, it's still rather unknown in general.

While I understand (and share) your frustration, I don't think this is the right attitude to express here, and if anything it's just going to (understandably) scare people off from using it.

I personally really wouldn't want the Nix community to end up being "those people who roll into any thread about package management yelling at people". Thankfully, for the most part that isn't the case right now, so let's keep it that way.

On the topic itself: yes, I think that everybody who's working on a package manager could (and should) learn from the work that's been done on Nix. Particularly the LISA paper[1] is a good and accessible read on why Nix works how it does, and what problems it solves that you might run into with your package manager project.

That having been said, that doesn't mean that there cannot be competing implementations. The important thing is ultimately the package management model, not the implementation, and it's no secret that Nix's implementation of the model has quite a few usability and documentation issues. Those might be easier to solve in a new implementation of the model.

In the end, everybody's working towards the same goal of better package management. So let's default to a constructive attitude towards other projects in the same space, rather than a tribal war.

[1] https://nixos.org/~eelco/pubs/nspfssd-lisa2004-final.pdf

I am a tiny bit confused. The readme says

"you don't have to figure out whether to use Pip or Pipenv or Poetry to manage your Python packages or wade into the Cabal-versus-Stack holy war in Haskell-land"

Python is on the list of supported languages, but Haskell is not...

Haskell support is planned. The idea is to cover as much surface area as possible. Help would be appreciated too.

I'm really grateful for the explanation of Project Package Managers in this. I've never come across that term and it makes the idea of per-language package managers make a whole lot more sense to me, as long as they are used that way. Without that concept, I lean much more towards using system packages. I'm going to keep Project Package Manager in mind from now on.

This is awesome! I started a wrapper around system package managers because of similar frustrations (pull requests welcome!):

https://github.com/krupan/package_commands

I really don't know why all package managers don't use the same simple commands.

Mental models of the creators

What are the advantages over conda?

I would imagine the advantages are more languages than R and Python.

Use OS packages instead.

And on OS X and Windows?

Does repl.it provide OS X/Windows backend servers?

No not yet. And it hasn't been requested all that much but UPM works on OS X and I'm sure it's straightforward to make it work on Windows.

Actually, UPM fully supports Windows, as far as I know. You can install it directly as an executable, or via Scoop.

Right should I package it as deb, RPM, tgz, deploy, p5i, pkg, msi, msix, appx, apk, aar, ipa, ISMP or do you have any other favourite OS package format in mind?

As fun game you can try to map those package formats to their respective OSes.

To be fair, isn't deb/apt already open source? If you don't know it, check out the "competing standards" xkcd.

If I was a FOSS dev, I'd say "screw it; just let people pull the git and package it however their distribution wants it".

Not every company wants to deliver open source packages, nor everyone uses git.

I mean, for 99% of the packages in most distribs, that’s how it works. Someone makes a cool binary, someone else packages it up and submits it to their favorite distro’s package manager.

No, you only package what you need.

very few (if any) will install as user, or install to a specified prefix for per-project versions of installs.

That doesn't work unless you can assume that every developer is willing to install all the dependencies of your project globally, with no control over what versions are used.

...which is how the majority of software in the world is deployed, outside of the HN bubble. There are many statistics on this.

But it's not how it's developed. It's not uncommon that you need to run multiple different versions of the software you're working on, each using different versions of its dependencies.

Containers are great for testing on a distro you don't use.

Most OS packages are out of date, putting new features or libraries out of reach of most people using them.

Not really.

Reply on news.ycombinator.com