RVM 2.0: Convert 20,000 lines of shell script into Ruby [Fundraiser]
bountysource.com
bountysource.com
I assume that if you don't know (like me), you're not in the target audience of people willing to chip in to get the conversion done, but it still manages to frustrate me.
To answer your question, RVM is a version manager for Ruby. http://rvm.io/
That's not even thinking about all of the service they've provided to our community already.
I would argue it's on the low side, I sincerely hope he'll go over it.
1. http://blog.zenspider.com/blog/2012/09/ohmygems.html
2. https://twitter.com/garybernhardt/status/388758868559290368
$100/hr is a very conservative estimate for an hourly rate for a senior level Ruby engineer that can read and understand shell code and implement clean, well documented code.
500 hours also seems very reasonable given that they'd be rewriting 20,000 lines of Bash. Not an easy task.
https://docs.google.com/document/d/1xW9GeEpLOWPcddDg_hOPvK4o...
Your question is covered under "Bootstrapping"
> Bootstrapping > Using ruby should be possible thanks to binary rubies, in worst case we can fallback to static build or use JRuby. The binary Ruby (or static or JRuby) will be installed along with RVM 2 using very simple bootstrapping script written in shell most likely pure SH to provide compatibility with as much systems as possible
I had similar thoughts. The fact that RVM is implemented (currently) in shell means you don't need Ruby installed. It does, however, expose your tool to all the idiosyncrasies of someone else's shell environment.
I sit in #rvm on Freenode from time to time and the vast majority of trouble pings are related to differences in environment. That is to say, if you run a standard (as in, not heavily customized) environment, you won't have much trouble. There are a large number of users who really like to tweak their shell though, so RVM has to face the issue head on.
I think this is what has lead the team to the "bundled" approach. It sounds like RVM 2.0 will be a full-blown application, rather than a collection of scripts. The choice to build the application in Ruby, as opposed to something like C, seems obvious when you consider what it is. Ruby folks want to write Ruby.
I guess many people use rvm to install ruby and I always recommend newbies to try it out for that purpose.
But recent criticisms of ruby made me wonder if that really is the right approach. Maybe it would be better to put more effort into creating top-notch packages for the major linux distros (and, say, homebrew). And to work with the distributions on the stable packages, while offering official repositories to get the latest versions.
rvm could still play a role in this scenario as a tool to switch between versions, isolate gem installations and so on, offering a unified interface that hides the platform differences of the distros.
I would certainly donate to such a cause.
https://github.com/phusion/passenger_apt_automation
https://github.com/phusion/passenger_autobuilder
And this is just one software package. Imagine at least 10 Ruby interpreters and versions (all versions of MRI 1.8, 1.9 and 2.0, all versions of Rubinius, all versions of JRuby). Now multiply that by the number of popular platforms. That's at least 13 (2 supported Ubuntu LTS versions, 1 latest Ubuntu non-LTS version, 2 supported Debian versions, latest 2 OS X versions, Windows, Red Hat, Fedora, CentOS, Arch, Gentoo). You'd end up with at least 100 different packages, and you'll have to test each one of them.
That's going to cost much more than a one-off campaign of $50000.
Of course you are right, and I am well aware of this. After all, that is the reason it has not been done before.
Still, a crazy amount of effort went into creating and maintaining rvm, rbenv and the myriad of other tools we now have. I cannot help but wonder where we would stand now had all of this instead been directed towards great packaging.
> You'd end up with at least 100 different packages, and you'll have to test each one of them. That's going to cost much more than a one-off campaign of $50000.
I am not quite so pessimistic. What I am daydreaming about would involve the distributions and their package maintainers. They already ship ruby right now. The packages are just not in optimal shape and oftentimes not up to date.
But one can surely build on the foundation already laid out by the distributions and work with them to hand off some of the work.
Still, I know this would be a herculean task. And not only a technical challenge, but a social one as well.
also check the update with schedule - https://www.bountysource.com/fundraisers/489-rvm-2-0 - "2 months: Work on building packages using RVM 2 (so it deprecates itself)"
The fact that no platform got it right for Perl, Python nor Ruby points to a huge difficulty...
Few people compile Python from source in the way that is common for Ruby. People do use pip/ virtualenv, but those are more the equivalent of bundler/ RVM gemsets-- they manage libraries, not entire interpreters. It's a little confusing because virtualenv appears to have its "own" Python, but really it is a symlink to the (usually) system interpreter managed by the distro.
The major distros have representatives who are active on python-dev in working out any issues that may make integration difficult between Python and the distros.
Does it? From my limited exposure to python I remember that e.g. recent CentOs comes with python 2.6 only and installing any never version requires you to manually compile it while being extra extra careful not to touch the system python since otherwise yum dies.
Ruby works just well on that level - as long as the system ruby is fine for you, all is settled. For the rest of us that need a never/other ruby there's rvm/rbenv/chruby.
I switched to rbenv years ago, never looked back.
This breaks other software in confusing ways. For example, I've had cases where after installing it, I could not view man pages for any programs because installing RVM modified broke my $MANPATH. In other cases, things that were failing for other reasons started producing confusing errors due to RVM doing sketchy things like replacing cd with a shell function.
RVM tells the user very clearly that it has modified the user's dotfiles.
And how many people do not want to modify the dotfile? Which user thinks, "oooh, modifying dotfiles is so evil, I'll just type in this command every single time I open a terminal"?
At the end of the day, RVM just inserts a few lines to the dotfiles. What's the problem with that?
Actually, chruby does not modify my dotfile for me, and I thought that modifying the dotfile is optional. I figured I'll just manually source chruby.sh every time I want to use it, just to experience how it's like. And then I ran into issues like this: https://github.com/postmodern/chruby/issues/213
Thanks but no thanks. I'd rather have the software modify my dotfile so that it can give me a pleasant user experience.
> For example, I've had cases where after installing it, I could not view man pages for any programs because installing RVM modified broke my $MANPATH. In other cases, things that were failing for other reasons started producing confusing errors due to RVM doing sketchy things like replacing cd with a shell function.
Never seen these problems before. Have you reported these problems? Have they been fixed in the latest version? If these problems are fixable then why throw the baby out with the bath water?
As for replacing cd with a shell function: it's because it's one of the few ways auto-switching can be implemented. They found in a user survey that 80% of the users want auto-switching, so who are they to say "no" to their users?
What problems have you experienced because of this? chruby's auto.sh implements the same functionality by installing itself into the shell's DEBUG hook... but is that really any better?
I don't know. I don't use Ruby or RVM anymore.
> Which user thinks, "oooh, modifying dotfiles is so evil, I'll just type in this command every single time I open a terminal"?
I did. And it's not "every single time I open a terminal", it's "the once in a blue moon that I need to work on a legacy Rails app".
> They found in a user survey that 80% of the users want auto-switching
Obviously people who choose to use RVM are going to be people who like the way that RVM operates.
> If these problems are fixable then why throw the baby out with the bath water?
The bath water is the specific bugs I ran into. As you say, those may fixed and aren't the point. The baby is the general practice of automatically injecting things into dotfiles and messing with shell builtins. We should throw that away because when it fails, it leads to "spooky action at a distance" bugs that are hard to understand and that break totally unrelated things (the user is in a shell session where they aren't doing anything related to Ruby).
And they are simpler, from a technical point of view. But unless you, the user, is very skilled at Unix and knows exactly how environment variables, shell sessions, bashrc etc works, it's going to be more confusing. RVM is more complicated because it does more, in order to make it easier for users. Notice that most of the people who like rbenv/chruby are exactly the kind of people who are already intimately familiar with the Unix environment.
I write C++ for a living and use the command line daily. Last time I used rbenv, I quickly gave up because I had to manually mess with $PATH, bashrc and other stuff that RVM always took care for me. I know how that stuff works, but the whole point of tools is to make my life easier. If I have to do so much stuff manually then why should I use rbenv? Back to RVM for me.
Have the RVM authors not fixed the problems you experienced?
Surely you must say no to some people... I can't take the risk that person might be me and then be left up the creek without a paddle.
Do you have an SLA? What if I have an issue in production and the core contributors are all sleeping? Can I wake you up? Can I pay you for the right to wake you up? Even if you are awake and willing to help on my schedule, can you do so reliably without breaking everybody else? As your complexity (and popularity!) grows, your ability to take responsibility shrinks.
EDIT: A quick Googling suggests that you do offer commercial support. Take my comments above more generally for the wider array of open source projects out there that real businesses depend on.
Why does Python still have a global interpreter lock after all these years, and why does it not have a JIT? One of the reasons is because it makes development easy, keeps the architecture simple and makes contributions easy. Writing a language implementation without a GIL, and with the latest state-of-the-art garbage collector and JIT, makes the code and architecture more complex also. The drawback of this is... that Python has a GIL and has no JIT.
rvm is complex both in implementation and in interface. The interface complexity comes from the shear number of things it tries to do automatically for you: interpreter versioning, gemsets, shell integration, plugins/extensions, devops functionality, and a bunch more! All of that is tangled together in one tool and you can't get any piece of it a la carte. Each of those things has its own plumbing commands and conceptual surface area. rbenv just fucking puts some shit on your path.
That's totally separate from an interpreter having inherent complexity in sophistication of algorithms and data structures. Personally, I think (as do the PyPy folks) that Python's design has some substantial complexities that make JIT-ing much more difficult than Lua, for example. If you choose a simpler language, you can get a simpler JIT; hence LuaJIT's success. Similarly, if you choose an even simpler language, with less mutability in core data structures, you can get a simple lock-free, multi-threaded interpreter.
This is what I think your Python example below misses. If Python is slow because it doesn't have a JIT and that causes someone's program to run too slowly, they can see that the problem is that Python is slow. Python's lack of a JIT would never cause "man traceroute" to stop working in a shell session where the user hasn't done anything related to Python and couldn't possibly guess that Python is the culprit.
I don't understand this mentality. rbenv isn't advanced shell scripting by any means. Every working programmer should have the minimal unix skills required to read and understand rbenv in its entirety in one sitting.
It should take no more than a week of concerted effort to learn your way around bash: You can read `man bash` linearly in an afternoon or two. For such a small price, you can gain hugely disproportional leverage.
> had to manually mess with $PATH, bashrc and other stuff
Personally, I wish Unix had a better framework for these sorts of environment/dependency/versioning needs, but altering $PATH in your bashrc is about as easy as it gets. It's not nearly as nice as a union mount plan9-style, but it's what we're stuck with, and it should take you no more than 5 minutes to understand and install rbenv. If not, then you should `man bash` and invest that time. Again, for such a small investment, you get a tool that is non-magic enough to trust on your production server too, rather than putting the rvm mystery box in to the wild.
I used to think that way too. It's so simple. Why don't people spend a little time to learn their tools? It's not that complicated!
Then, over the years, I realized that that's the wrong attitude. You shouldn't expect users to learn stuff no matter how much you think they should. They already have enough other stuff to worry about. They already have their hands full learning other stuff. And the reason why I thought it was easy, was because I already knew it. Expecting other people to go out of their way to learn the things you know is disrespecting their time.
Therefore, software should be easy to use for new users. Software should have a learning curve that's as low as possible. Software should not expect people to learn things that you think is important.
I am very firmly in the pro-usability camp, even for developer tools.
You said that rbenv isn't advanced shell scripting. You are right, it isn't - for us. However, that doesn't mean that it can be utterly confusing for a lot of people. The concept of environment variables are easy, until you get to the common edge cases. Did you know that environment variables are inherited on a per-process basis, so changing them in your shell does not affect already-running processes, and doesn't affect cron? Did you know adding environment variables to bashrc has no effect on cron, init scripts and the like? Lots and lots of people don't realize that. Do you want those people to give up in frustration? I'd say no. I'd say we should help those people with good documentation, and software that just works.
For example, at the Phusion Passenger mailing list, we regularly get questions from people asking why they get "command not found" errors. Apparently a lot of people don't understand what PATH is, or what environment variables are, or that sudo resets them. So we ended up writing this: http://www.modrails.com/documentation/Users%20guide%20Apache...
That might be true if you are going to use the tool precisely once and then never again, but this is your system's shell we're talking about. You're disrespecting your own time if you don't learn how it works.
> You said that rbenv isn't advanced shell scripting. You are right, it isn't - for us.
You're conflating the objective and subjective notions of complexity and difficulty respectively. rbenv is objectively simple and subjectively either easy or hard depending on your shell scripting experience. However, rvm is objectively complex and subjectively easy or hard depending on your Ruby environment customization needs. Programmers should prefer objective simplicity and leveraged familiarity every single time.
A small initial investment in learning your shell is significantly more valuable than a large delayed investment in learning RVM.
> Did you know that environment variables [...]
Absolutely everything you mention is discussed in great detail in the bash man pages. And even "in great detail" means that the ENVIRONMENT section fits on my 13" screen vertically at 80 columns and the INVOCATION section (which I found by searching for "bashrc") is no more than two screenfuls. That's a 10 minute trade for a lifetime of saving 5 minutes a piece on Stack Overflow visits.
Great. If you would tell the entire human population about this so that they don't ask these questions anymore, then I would be very grateful.
Until then, I'm going to assume that people don't, and don't like to, read documentation, and adjust my software to that fact.
Fine, but rvm has chosen the completely wrong way to go about adjusting the software to that fact! Rather than build a simple system that has an optional easy-mode layer on top, rvm begins with some crazy complex shell-builtin overriding functionality and works backwards from there. If at some later time I want to use the just put shit on my path mode, I'm shit out of luck.
And yes, so now I need to install ruby in order to install ruby?!
I do not understand why the[1] shell... which is a relatively small language... is so difficult for people to learn.
If you can find the time to learn Ruby, what stops you from learning a shell? What is the cost of ignoring it?
1. Preferably something resembling the proposed POSIX standard which might lend to portability. Bash is not necessarily the best choice unless every server you will ever use will be GNU userland/Linux kernel.
ps : I was conflicted I couldn't believe ruby's vm would be built on 20K lines of bash. <relief/>