Asdf – An Extendable Version Manager | Hacker News Reader

Asdf – An Extendable Version Manager

asdf-vm.com

238 pointsBerislavLopac96 comments

Not the ASDF (https://common-lisp.net/project/asdf/) I was expecting.

Looking up documentation when you have build problems in CL is now very hard. It was already very hard however, so no big deal.

Not the Advanced Scientific Data Format either.

https://github.com/asdf-format/asdf

Neither that ASDF, nor that kind of VM, despite the URL.

Yeah I'm not usually a "hey there's another project with that name" kinda guy but, build manager and version manager?

Isn't there some other part of the keyboard they could have mashed?

Sure there is. One of the proposed successors to ASDF the Common Lisp package manager is XCVB (https://common-lisp.net/project/xcvb/), the eXtensible Component Verifier and Builder.

Why is there no common place where people can see names that have been used already? It could make us avoid such nuissance.

Why don't you build one? The worst that can happen is that you learn the answer to your question.

There is. It's called Debian.

There is, google: "asdf programming". All the other projects named asdf pop up. The people who do not use google will not use a new common place either.

It's not asdf movie (https://www.youtube.com/watch?v=IYnsfV5N2n8) either

I've been using it and has contributed to it, but I wish it supported out of the box GitHub-based release binaries as 90% of my code for different vendors was relatively the same, so, instead of having multiple identical repositories, I created one, which uses introspection [0].

I wish this was available out of the box to handle literary 90% of the tools.

Also, I typically pair it with direnv [1] for even more magic.

[0]: https://github.com/Banno/asdf-hashicorp

[1]: https://direnv.net/

Thanks for sharing direnv, I didn’t know about that but I’ve wished for it before. Great find :)

It's a gem, really.

I use nix/direnv/lorri to manage all my projects. It’s easily the best def experience, once you get over the learning curve for nix.

Thank you for asdf-hashicorp. We use it at Skillshare.

Thank you! It's nothing special.

Thank you for asdf-hashicorp. We use it at HashiCorp.

You're welcome!

Direnv/asdf is crazy fantastic. I was afraid asdf and direnv would fall into the venv/conda hell that is python, but it. Just. Isn't.

I've also written a handful of plugins for asdf, and this is exactly my experience also - I even used the asdf-hashicorp library as the basis for https://github.com/vmware-tanzu/asdf-carvel

Seems like a lot of duplicated work across all these plugins

Can this replace the broken and horrible tools of rbenv, pyenv, and pipenv? I shudder whenever a new version comes out what it does to my shell when I have a broken version (included as the latest from the output from pyenv globals or some such)

I'm not understanding what you mean about *env. I use rbenv and I'm a maintainer of crenv for Crystal, I never have any issues with how they are implemented.

Can you be more specific? Maybe something's broken.

pyenv and pipenv are two very different things. pipenv is desperately broken, and should be avoided; Poetry is one good alternative. pyenv, on the other hand, is great.

Seeing all the love for direnv makes me feel like I'm the only one taking crazy pills.

Direnv breaks referential transparency of the shell. I heavily use my shell history to riff on commandlines I've entered in the past. If direnv is in the mix, my shell pipelines sometimes have to have `cd` in them, often in subshells because that works better for me than pushd/popd.

pyenv/rbenv/asdf do the same thing, but you can have a "global" version pinned and do all your shell-related package installs there, so it doesn't end up being that bad. Though, this still breaks when I happen to be sitting in a directory on my system that has a .python-version file. All of a sudden my "global"ly installed python packages don't work. I greatly prefer tools like pipx / pipsi for this purpose (is there a language-agnostic pipx-alike? or is that nix?) because the shims they install usually work no matter what my shell environment/CWD is.

.... Maybe most of my ire is because my only exposure to direnv is because at least one repo at my day job uses it. For me it's not so much "customize your environment any way you want", it's "oh did you run that command from projectdir/foo instead of projectdir/foo/bar? That doesn't work". Debugging the interactions between people's individual poetry/pipenv/pip/rbenv/direnv/etcetcetc setups and project-based setups is the bane of my work life.

Although I agree in general, direnv is pretty transparent. For example, you need to approve any change in .envrc and it's pretty verbose. It also solves another big issue for me - I no longer have to install globals like with "npm i -g"and I can use "pip install --user" without trading off convenience and using weird relative bin paths!

You can just not hook direnv into your shell, and use `direnv exec` kind of like pipx.

Tools like asdf, direnv, nix-shell, etc. just encapsulate the environment and help set up some guarantees. The referential transparency of shells is something that these tools help enforce, if anything.

I agree that frequent jumping between the fragmented environments is pain point in software development these days. That's due to a lack of tooling to support the new workflow, in my opinion. I hope enough people feel this pain that we see some solutions.

Having an expressive shell like https://starship.rs helps keep you oriented as a sort of HUD. Nix is definitely a life-saver, but you can probably roll your own nix-a-likes. Encapsulate all the "global" precious tools, hardening them against changes in the "local" shell environment. Whether through wrapper scripts, containerization, or what have you, the building blocks are there, the "best practices" are still being created.

Magic. This is not what I'd like to see around my code and my systems. I'd prefer everything about them to be transparent and explainable.

(My favorite worth-1000-words picture about that is e.g. here: http://www.tshirtvortex.net/free-magic/)

I just woke up to find this thread on home page and read all the comments.

Thank you for the feedback. I’ll ping the team to let them know about this thread.

The plugins were kept as separate repos - like Heroku Buildpacks, because I never had the time to vet/review them. I had written plugins for Ruby, Node.js, Erlang and Elixir because those are the ones I wanted. I did not expect the project to be active this long or have these many contributors, maintainers and users.

We’ll bring back the readme in the repo with usage instructions.

P.S: Author here. Not an active maintainer except helping clean issues

We use it at GitLab since a few months in order to manage our development toolchain [0]. Before we had people using rvm, rbenv, nvm, homebrew, system packages and everything in between. Supporting our engineers and non-engineers has been a lot easier since asdf, and the whole version manager related issues calmed down a bit.

The only thing I miss from homebrew is the pre-compiled stuff like e.g. Postgres or redis

[0]: https://gitlab.com/gitlab-org/gitlab-development-kit/-/merge...

For Postgres & Redis version management, have you seen DBngin? It supports MySQL, too.

https://github.com/TablePlus/DBngin

Looks nice, but we’d still need to support Linux too. And CLI for the win!

Eventually moving our Postgres, Redis, etc. to docker is also a solution. Unfortunately node and ruby take a performance hit if you mount them into containers and update loads of files.

Have you used or considered nix before? I’m looking into both right now and wonder what the advantage of asdf is, as nix seems to be the more robust option.

I think we have looked into Nix briefly. It seems great. I wonder how well it works cross-platform or if it forces us to either maintain a Darwin and Linux version, or using the Linux version in a VM.

The whole idea seems to be that you don’t need to care about the platform. You use the same .nix file to start a shell with a custom environment on Linux, Darwin and Windows WSL. Even aarch64 is fully supported and all binary packages are built for it as well, so even on the M1 Macs it should just work.

Granted, I didn’t try any of this yet, but the more I read about it, the more I want to. You can even build docker images from the .nix file so the CI/CD pipeline will have the exact same environment as all the dev machines.

I've been using it since 2017 for an Elixir/Phoenix backend, React frontend and PostgreSQL database. It works very well and spares me from having to use three language managers (one for Elixir, one for Erlang and one for JS) plus a docker container for PostgreSQL. I was pleasantly surprised that it manages the database too.

Mind elaborating on how it manages the database? That sounds useful! Does it install and run the db for you or something?

It manages the version of the database that you run. https://github.com/smashedtoatoms/asdf-postgres

I'm assuming the author was referring to this plugin: https://github.com/smashedtoatoms/asdf-postgres

Sibling answers link to the plugin with the details.

The general idea is that it installs the binaries and the data into a subdirectory of ~/.asdf, then sets PATH and the other environment variables to point to there. Then pg_ctl start starts the database.

Each project has its own .tool_versions file so you can have different versions of the same database running in parallel. I was doing that with docker containers but this is easier.

My install of asdf at some point broke so badly, I could not get anything to run. I wish I remembered the exact trouble, but it was bad enough that I just un-installed it in favor of `nvm` for node versioning, and never looked back.

n is still the simplest node version manager I've come across [0]. Maybe worth checking out if you're looking for an alternative.

I also use the companion n-install [1] utility to get setup very quickly.

[0] https://github.com/tj/n

[1] https://github.com/mklement0/n-install

Would be nice if, instead of a poem, the main page showed a representative example so folks can get an impression of what it's actually like.

My greatest asdf wish (well, I guess not greatest since I've never taken the time to submit a patch...) is for supporting pre-compiled Erlang (see https://github.com/asdf-vm/asdf-erlang/issues/120). It's often an adventure getting the whole team updated whenever we bump our Erlang version.

I am using it for python, erlang, elixir and nodejs. It works great on MacOS so far. I used to use a mix of homebrew versions of those tools and some language-specific install scripts like kerl or pyenv.

One of my favorite things is to quickly start a new shell with a specific version to test something: 'asdf shell python $ver'.

I did try direnv as others mentioned here, but I found it a bit too magical and eventually abandoned it.

It's growing in popularity in the Rails community (probably because Rails now requires both Ruby and Node). Previously I recommended chruby in my installation guide for Ruby [0] and Rails [1], but after doing an informal survey of Rails users, I now show how to use asdf.

[0]: [https://mac.install.guide/ruby/index.html] [1]: [https://learn-rails.com/install-rails-mac/index.html]

Actually I should add, I wrote a bit more nuanced recommendation in the installation guide. I say:

Install with Homebrew if you’re building only one project with Ruby (especially if you are a student learning Ruby). If you’re a solo developer and you're juggling multiple projects that can't be updated all at once, use asdf or chruby or rbenv. Choose asdf if you're using multiple languages such as Ruby, Node, and Python; otherwise chruby or rbenv are fine for just managing Ruby versions. Finally, use Docker or Nix if you’re on a team with a complex project environment (for example, Ruby, Node, Redis, and PostgreSQL all in one project).

"Please head over the documentation site" is a link back to the site. The ballad links to an old repo (fortunately Github redirects it) The all-plugins page 404's.

After giving up on the website I look at the code. It's a shell script. Wonderful.

I'll stick with Nix any day. Or maybe conda.

The all-plugins page works fine for me, and is no Shell script: https://asdf-vm.com/#/plugins-all

Are you browsing with JS turned off? (The page does some JS tricks)

Indeed, JS was off as it usually is.

Do you have JS enabled? It does seem to be one of those pointlessly-JS SPAs. (Yes, I'm judging them for this choice.)

Yeah considering it doesn't work on windows, the more general tool nix, which has good integration with direnv works very well.

I wanted to see what the differences between asdf and nix were, and found a pretty lengthy article[1] that explained it quite well in the first few paragraphs. asdf is better than nothing, and better than multiple version managers for each language in one project, but it is a far cry from the portability and reliability of nix.

If you wanna do something quickly to ease the pain of version management, asdf is there, helpful, and ergonomic. But if you want to make version management an issue of the past and fix it for good, nix is the way to go.

So they both have their use. nix just feels more proper.

I'd be interested in that article - be great if you could re-up the footnote :)

Oh damnit, completely forgot. Seems like I can’t edit my comment anymore, so here you go: https://ejpcmac.net/blog/about-using-nix-in-my-development-w...

asdf makes my life so much easier seriously, it's amazing. Try it!

I have a .tool-versions file in code repo that lists what I need for a project.

    elixir 1.2.3
    erlang 1.2.3
    nodejs 1.2.3

Whenever I `cd` into the folder, asdf automatically uses the right versions for me. I don't have to worry about anything. Razer sharp scalpel of a tool.

The only issue is (was?) that you need to preinstall these unless this has changed in the past year. When I integrated it with direnv [0], I made the missing ones autoinstall.

[0]: https://direnv.net/

Correct, you must have them installed so the `cd` into the directory auto switches. But that's a simple `asdf install elixir 1.2.3` boom you're set

Or just `asdf install` - it will handle all the missing versions.

Yeah, that's what the direnv script was doing. You cd and forget!™

Couldn't agree more. asdf has made working across multiple computers and projects dead simple.

:heavyplus:

.tool-versions means everyone on the team knows which version(s) to use for a project, for anything outside of docker. Auto-install is an awesome feature, too.

Smart that it’s based on rbenv and the many spin-offs (pyenv, nodenv etc).

The name is off putting to me though, because it’s meaningless. Maybe I could alias it to something like xenv and pretend it didn’t bother me.

No way! asdf is full of vast meaning. :) http://asdf.com/whatisasdf.html

Asdf is fantastic for forcing versions of elixir, especially since no native version control exists for the elixir vm itself. Tools like this are developer friendly and also easy to introduce and share - I've had nothing but fantastic experiences with Asdf.

ASDF is great. I've been using it for four years now with no real complaints.

It's made collaboration with contractors much easier to just be able to drop a .tool-versions file into a project and know that regardless of what OS each of us are on, we'll have the same language versions.

We started using this for elixir/erlang and quickly replaced all the other version managers company-wide. The only issue we've really had is that sometimes people don't understand reshim and when they need to do it.

When does one need to use reshim?

In my experience, after installing a version of nodejs, whenever I install a global npm package I have to manually reshim (`asdf reshim nodejs`) so that bash will be able to locate it in $PATH.

So, this in contrast to docker? Is this for development or for deployment?

It's for switching versions of your local CLI tools quickly.

> asdf is a CLI tool that can manage multiple language runtime versions on a per-project basis. It is like gvm, nvm, rbenv & pyenv (and more) all in one! Simply install your language’s plugin!

For example:

    $ asdf global python 3.6.2
    $ python -V
    Python 3.6.2

    $ asdf local python 3.9.1
    $ python -V
    Python 3.9.1

"asdf local" stores the version to use for the current directory across shells, which is neat. "shell" is for the current session, and "global" is for the system.

The supported CLI tools is extendable via plugins: https://asdf-vm.com/#/plugins-all

No, it's one system to have in your shell rc - that does the work of nvm, pyenv, rbenv etc.

Main benefit is you no longer need N half-baked utilities fighting over your path and shell completions, just asdf.

And plug-ins mostly re-use other tools, so it's both simple to set up and mostly avoids duplicating effort.

Yeah, looks this would be an alternative to setting up a development environment with Docker or a VM or whatever.

Tried asdf few months ago but couldn't get warm with it. Went back to nvm for node and just venv/ for Python. Anyone went back too and if yes, why? Thinking of giving it another try.

Yes. I tried it, ended up writing my own node version manager inspired by chruby, then went back to asdf and have been so glad I made the switch back. It's performant, it makes it easy to manage all the things where I care about using a specific version so easy, and the maintainers are super responsive and engaged.

FWIW and a note to myself before I check it out a third time:

I just checked it a second time and recall now what was turning me off: I found the installation of the node plugin and node super tedious and too much overhead[1], it's just once but yeah, didn't like the the UX (eg reshim), then asdf always forgot my current node version and finally, I was missing a quick ls command for checking out installed node versions and versions/lts-versions available remotely (nvm ls and ls-remote ) without having to checkout the node website or repo.

I cannot speak for other languages but I found it for node rather a bigger step back and for stuff like Mongo—it's in their plugins!—I just use Docker, IDK but asdf feels just wrong for things like Mongo. I usually use node also from the node Docker image and have node versions only locally installed for stuff like coc-vim or when quickly needing the node REPL but anything serious runs in a container with a volume mapped to the project folder.

So, I guess it's rather for languages that do not have a proper version manager and/or folks not into containerization.

[1] https://github.com/asdf-vm/asdf-nodejs

Could this tool be used to create a meant a software bill of materials, or is there more work needed for that effort?

> A bill of materials or product structure (sometimes bill of material, BOM or associated list) is a list of the raw materials, sub-assemblies, intermediate assemblies, sub-components, parts, and the quantities of each needed to manufacture an end product. [0]

[0] https://en.wikipedia.org/wiki/Bill_of_materials

I've had mostly good experience with asdf except last month when I tried to get it working on an m1 Mac. Eventually gave up and just brew installed everything (erlang, elixir, nodejs, java. New work laptop that only needed one version of everything for now). There's an open issue on the project for support.

Later on I ran into still more problems on the m1 and convinced a project manager to take it and got an Intel Mac instead and back to blissful working asdf.

If you are interested in an extensible distribution independent package manager, then you might be interested in Guix: https://guix.gnu.org/manual/en/html_node/Package-Management....

I've been using it for Ruby and Node.js for years and never had issues with it. Great tool.

Does anyone know a good way for asdf to inform which versions are installed in container?

I feel like I keep having to hack together ways to do the same thing when building containers for apps build with asdf. (Github actions, dockerfile, heroku, etc)

I don't think using asdf in your container builds is a good idea tbh. You should pin your container to a specific version, or perhaps set the base image version by parsing the .tool-version file.

Yeah, I didn't write that clearly.

We have this great .tool-versions file defining the version of ruby, node, yarn, etc. It's great.

Then we setup github actions to run linting/tests/etc, and we have to define those same versions either explicitly in the .yml or via some hacky shell commands to pull version info into a RUN curl...

It would be great if there was a little more standardization around language versions. Or maybe there is a secret convention that I just haven't heard of yet.

Been using it for kubectl, kops, terraform, and a few others. It's really great!

So how does this tool compare to module[0]?

[0]: https://modules.readthedocs.io/en/latest/module.html

I introduced ASDF at the company I currently work at and it spread like wildfire to every team.

Having a single version manager that manages the versions for numerous tools across different teams has been invaluable.

I use asdf for python, golang, and hashicorp tools. Having a single tool instead of a separate "env"/"vm" tool for each new thing is fantastic.

My team is using this, and I have no complaints about it.

Used this recently for installing Erlang and Elixir on my Debian system where apt was out of date.

As a Colemak user, where do I submit a PR to rename this project to Arst? For usability sake.

Are you kidding to change the name for some niche users?

Use alias.

Although I don't like the name as well as it's only easy to type but nothing really meaningful.

simple and straight forward using this for a while now for BEAM.

The webpage doesn't load with JS disabled, and when you enable it, it just displays a bunch of text and some hyperlinks. This is beyond ridiculous.

Conan (https://conan.io/) can do the same thing, and I think it is quite a lot more versatile at doing things like that.

I know that Conan claims to be a generic packager and tool manager (https://docs.conan.io/en/latest/howtos/other_languages_packa...) but is anyone using it for that?

asdf seems like it has much wider support

Reply on news.ycombinator.com