Refining Ruby | Hacker News Reader

Refining Ruby

blog.headius.com

168 pointsjballanc54 comments

The most convincing argument as to why not to include refinements in the Ruby spec comes from alternative implementations like JRuby and Rubinius. Method dispatch will need to be vastly more complex (and have scope-limited caching) to support this feature. Much, much slower. Folks using these implementations are still on the fringe, but it seems wrong to add a feature to Ruby that makes both more complicated code and slower running code.

Wasn't Object#source_location added in 1.9 to help folks understand and find method definitions in code? Shouldn't we simply test our code to ensure it functions correctly and not force additional semantics all the time to fix a rare issue?

> Shouldn't we simply test our code to ensure it functions correctly and not force additional semantics all the time to fix a rare issue?

The problem is you can't. If library A and B define String#to_param differently for internal use you can't use them together, while both of them are perfectly correct on their own.

I've been writing Ruby professionally for six years and this has never been an issue for me or folks I work with. Can you give a single example where you found a conflict in libraries that broke your application?

I ran into a nasty bug a while back which came down to a conflict between two JSON libraries in their definitions of #to_json on core classes.

I have seen issues quite a few times, and they are serious headaches to fix when they come about. I've had libraries implement Rails utility methods slightly differently, or conflicting implementations of monkeypatches on S3 libraries.

While it's not incredibly common, disrespecting namespace concerns is a sure road to problems.

Rake::RemoteTask (a dependency of Vlad) defines a bunch of stuff on Kernel, including #role. For whatever reason, if an ActiveRecord model includes role as a property, things break in weird ways.

Reasonably easy to work around by aliasing/undefing Kernel#role to #server_role or something, but amusingly that would be something that could be fixed by refinements (though I'd rather see something more like Thor, that's not run toplevel).

I think the same. For 99% of Ruby programmers speed is bigger issue than monkey patches conflicts.

Maybe refinements should evolve into some MRI diagnostics mode.

http://bugs.ruby-lang.org/issues/4085

Headius technical comments on ruby tracker are a must-read.

As my personal take, I am appalled at refinements. I think this is a bad idea solving the wrong problem.

Thanks for linking this. I'd only glanced at refinements so far... reading these comments and example code makes my head hurt. I wouldn't have thought "solving" monkey-patching with something even more convoluted and confusing was even possible. My favorite code example is from tenderlove:

  class Foo < SomeParent
    def baz(str)
      cached = str.camelize
      ary.map {|name| cached + name}
    end
  end

Could have a completely different meaning than this code:

  class Foo < SomeParent
    def baz(str)
      ary.map {|name| str.camelize + name}
    end
  end

/lights hair on fire

I wish 2.0 had focused on speed, not a bunch of features that working rubyists really don't care about. Esp. not ones this poorly thought out (you'd think given 2 years discussion they would have abandoned the idea).

Speed and Memory usage. Both needs a huge improvement. And i could not understand why they are even trying with mRuby. ( I mean have they not heard of a thing called Lua and LuaJIT? )

Hopefully a lot of the tweaks and refinement will come from now to Final 2.0 release.

> And i could not understand why they are even trying with mRuby. ( I mean have they not heard of a thing called Lua and LuaJIT? )

LuaJIT is pretty much impossible in a language with as complex semantics as Ruby (it's probably impossible in Python already, and Ruby is significantly more complex)

The decision making in the Ruby world seems to be fairly centralized and not very inclusive. To see that one of core devs of one of the biggest Ruby implementations thinks it's a bad idea is really confusing to me. I really wish there had been more discussion about it before it happened.

Cause now not only do we have that confusing feature which very few people seem to like, it also underlines a worrying trend where the different parts of the Ruby world don't reach a consensus before going in radical directions.

No one summed it up better than Brian Ford at his RubyConf talk really: http://www.confreaks.com/videos/1278-rubyconf2012-toward-a-d...

It's a must-see.

Refinements are not yet a part of Ruby 2.0 and their implementation in Ruby 2.0 is still under discussion. As Akira Matsuda pointed out @ the same RubyConf: "Warning: Refinements are experimental and may be removed from Ruby 2.0!" [23:15]

http://www.confreaks.com/videos/1275-rubyconf2012-ruby-2-0-o...

While I enjoyed most of Brian's talk, I wasn't particularly convinced by his dislike of refinements. Essentially, it boiled down to "this feature is hard to implement, it hides complexity, and it isn't well specified". The first and third arguments shouldn't matter to us nearly as much as the middle one. And I fail to see how namespacing monkey patches can do anything but decrease and contain accidental complexity (in the same way that method dispatch and polymorphism are better at limiting complexity than explicit conditionals).

The benefit to refinements is in preventing code conflicts from a shared namespace (like monkeypatching a core class).

The downside is it becomes even _harder_ to figure out what code is being run just by reading it. Right now, you just have superclasses and included modules, potentially gummed up by method_missing. Now, you need to also pay attention to refinements included by code that _calls_ the code you're looking at, which may change the way it behaves.

Unless used with great care, it's going to create a nightmare debugging situation. And with some of the code I've seen, it'll happen.

They have done minor improvements to both speed and memory usage in 2.0.

http://igor-alexandrov.github.com/blog/2012/11/05/yet-anothe...

how can a problem be "wrong" ?

I don't much like refinements, but conflicting monkey patches from different third party libraries _has_ been an issue over the years.

The ruby community is not even the first one to notice that, there is plenty of literature on "selector namespaces" and "classboxes" from the smalltalk crowd.

From a language designer's perspective, conflicting monkey patches seem like a problem. But if you ask Ruby users I bet 90%+ would say speed and memory usage are much bigger problems.

It has been an issue, but the much less painful solution is to be more careful about monkey patches, and employ other patterns whenever possible.

In my eyes, monkey patching should only ever be considered:

1. To fix outright bugs or nasty performance issues, where the monkey patch should not have other side effects.

2. In application code, never in libraries (except as a library explicitly providing monkey patches to an application, but never as a requirement for a library to work).

3. In adding new methods, except for case 1.

Of course there'd be exceptions, but very little monkey patching I see in library code is necessary or worth it.

Inside your library, you have plenty of ways of avoiding the need: Wrap objects; convert objects; use helpers. Yes, it might not look as perfectly smooth, but I'd take that over trying to reason about code that relies on different sets of refinements in different scopes any day..

When it comes to monkeypatching, this is one thing that JS and Ruby got wrong. C# got it right. It'd be great if you could monkeypatch in JS/Ruby in a module level scope vs a global scope. Good to see that Ruby is taking steps to remedy this. Too bad my favorite language, JS, hasn't yet.

Just write your own JS interpreter inside of JS and make sure you execute your functions inside your own interpreter and not the one provided by the browser... problem solved.

C# had an easier time here, though, because it's statically typed. Extension methods rely entirely on static types and static binding.

Getting normal method call syntax while still being lexically scoped like extension methods without opening the monkey-patching can of worms is still an open research area as far as I know. I know of classboxes, a proposal for something similar to ES Harmony (that they ended up passing on) and Ruby refinements, and that's pretty much it.

I had a good ten year run with Ruby and I still enjoy using it for quick utility scripts but I will never again voluntarily build any large systems in a dynamically typed language. The benefits of static typing for performance, correctness, and tooling are just too valuable.

I agree with the intent, but unless you're referring to something else (could well be), Extension Methods in c# aren't monkey patches. They're just semantic sugar. They didn't require new runtime support, just compiler support.

Which is a neat way to go about it.

Yes, reading this article makes me feel better about how it works in C#. (I think you mean syntactic sugar rather than semantic sugar though; the semantics stays the same, but the syntax is a little sweeter.)

Absolutely. My mistake.

While I run into unexpected monkey-patching from time to time (i.e. just about anytime I include an "active_" library from Rails), I can't remember the last time it's screwed me over. But I can see the potential...what have been the most prevalent cases of conflicts for day-to-day Rubyists?

Merb & DataMapper's integration with Rails 2 was seriously hindered by ActiveSupport's core monkey patching, coupled with how ActiveRecord was inextricably linked with other Rails components.

As a consequence, it really took Rails3's clean up of ActiveSupport to allow for other ORMs and libs to integrate in a safe manner.

I'm not 100% certain about the way that refinements currently work (mainly because I haven't had time to dig into the proposal), but the case that wycats makes for refinements amount to defensive coding. When you chose to include a lib, you should be able to control whether and how it steps on the rest of your app (and w/ refinements that can be done by specifying for other parts of your app, that it should/n't listen when someone tries to monkey patch its stuff).

One of the things that terrify me about refinements is that it'll just end up as a band aid for bad designs in cases where the vastly better solution would be to redesign without the need for monkey patches. They we end up with code that's potentially a lot harder to reason about out of pure laziness.

I had issues with libraries providing fixes for builtin libraries (I believe Net::HTTP), yehuda katz had his own example[0] with activesupport and right_aws, the memorable Chainsaw Infanticide Logger Maneuver[1] rant was about plenty of people redefining Logger's methods. I also believe one of the reason rspec changed their syntax is to avoid some monkey patching issues[2]

The bad thing is, most of the issues would be caught quickly if we could run stuff with -w, but nowadays nobody uses and you'll get floded with warnings from the libraries you use :(

[0] http://yehudakatz.com/2010/11/30/ruby-2-0-refinements-in-pra...

[1] http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/...

[2] http://myronmars.to/n/dev-blog/2012/06/rspecs-new-expectatio...

What problem does monkey patching solve? It seems to only add complexity onto the already complex nature of inheritance. What do I gain by adding camelize to String vs. using MyModule.camelize?

Some users think:

    "foo".camelize

reads better than:

    MyModule.camelize "foo"

Better examples:

    5.days.ago
    
    User.where(:age.gt => 5)

The `:age.gt => 5` was really a mistake. Since IIRC, I invented it, I think I'm qualified enough to assert that. ;-)

Something better might look like: https://gist.github.com/4113718

`5.days.ago` is actually harmful IMO. You get used to stuff like that, and then (if you're me at least), you basically forget how to perform basic date/time operations without it. Is +int on a Date referring to Day? Month? Year? What about on a Time object?

Custom percent-expressions would be a more elegant solution I think, allowing you to embed truly distinct mini-languages in Ruby: %date(5 days ago). Or %time(-12 hours), etc.

You could always overwrite modulo on classes that didn't need it: `Date % "5 days ago"`. You lose syntax highlighting that way though.

I guess my point is, there are other ways. I've put 5 minutes of effort into these alternatives. Monkey Patching can be a harmful crutch.

Though OTOH, these days I think Ruby would be entirely better off if it just included ActiveSupport into stdlib, and it could benefit from the optimizations available as a first-class Ruby library. It's basically defacto, and despite being involved with a couple different alternatives, I think that'd be best for everyone.

At the end of the day, I think if you want to make a strong argument for/against monkey-patching, it's best to leave ActiveSupport out of the discussion. It's so prevalent, it's hard to make a general case either way since pretty much everyone is aware of how to avoid stepping on it.

> Something better might look like: https://gist.github.com/4113718

Looks quite similar to how SQLAlchemy does it, although it doesn't use a block: http://docs.sqlalchemy.org/en/rel_0_7/orm/tutorial.html#comm...

> Custom percent-expressions would be a more elegant solution I think, allowing you to embed truly distinct mini-languages in Ruby: %date(5 days ago). Or %time(-12 hours), etc.

The issue there being that 1. it's going to become a pain to parse fast and 2. it starts looking a lot like genuine text, and thus people will want it localizable, thus bringing more of 1

ssmoot says he invented this, which is eminently readable but feels like being too leaky (what's the name of the deprecated gem that does that to AR already?):

    1) User.where(:age.gt => 5)

ssmoot says this is better:

    2) User.where { |user| user.age > 5 }

SQLAlchemy does it this way:

    3) query.filter(User.name == 'ed')

which looks so much like how AR3.0 + ARel does it (and seriously AR::Base should delegate [] to arel_table):

    4) User.where(User.arel_table[:age].gt 5)

and here's squeel's way:

    5) User.where{age >= 5}

FTR, Django QuerySet does it by parsing kwargs (convention is double underscore => dot, which allows to 'call' methods on fields (pub_date__year) and joined relations (group__name)):

    6) User.objects.filter(age__gt=5)

From a 'user' (i.e developer consuming the API) perspective I really like 1 and 5 because there is no redundancy (with which 2 is full of). Still when you're join()ing, you have to make ambiguous things explicit, and in that case 1 falls apart (you're not going to write :group_name, are you?), so while 5 is nice and allows for niceties like group.type, I am perfectly content with 4 because it's both stock Rails and quite readable, it's just that the full sized User.arel_table breaks the reading flow, and a bit redundant in the trivial case.

I'm not fond of stringifying stuff as resorting on parsing makes it less dynamic and more prone to abuse, and it we write strings we might as well write partial SQL.

Indeed ActiveSupport is a bad example because it extends by monkeypatching, but implements stuff that always apply (all strings are potentially camelizable, always, ever, and arguably integers can always be qualified with units) whereas Symbol#gt is nonsensical outside some specific scope.

The refinement situation is terrible because it performs poorly, leaks badly and makes things terribly inconsistent, especially with blocks passed around. I can't even begin to fathom the consequences of a block being called on an unexpected binding.

Your gist is exactly how sequel does it. However, I think having optional (!) syntax for extending symbols is absolutely fine as long as your library does not depend on it.

Regarding including ActiveSupport in stdlib, I see an opposite trend of extracting everything from stdlib into gems because it's hard to provides updates to anything in stdlib. Let's leave it as it is.

Btw, are you the original author of DataMapper?

Yes.

> `5.days.ago` is actually harmful IMO. You get used to stuff like that, and then (if you're me at least), you basically forget how to perform basic date/time operations without it.

that's not harmful - it's pretty useless to remember a worse way to do things, especially since you can always just pull in a dependency on activesupport or even paste in that piece of code into your own project and then continue using .days.ago.

That syntax only works against `now'. The mixins are a convenience for a subset of date/time math cases.

Among other things, MyModule.camelize doesn't provide an extension point for polymorphism. Consider a serialization library in which different classes should be able to have separate freeze/thaw behavior. It's possible to do this in current-day Ruby without easy collisions, but it tends to involve cumbersome manual prefixing or emulating method dispatch yourself.

Methods being namespaced in packages separately from the class hierarchy is something CLOS has that I miss in almost all the more Smalltalk-y languages.

MyModule.camelize doesn't support polymorphism (you could even call it procedural programming in a OOP shell), but something like Camelizable.new(my_string).camelize does. I'm still convinced that what Ruby needs is more object composition (and maybe a way to override default literal construction), and that refinements are a cannon to shoot a mosquito.

Camelizable.new(my_string).camelize does not support polymorphism, because it does not depend on the type of my_string.

Which means if I want to add camelizable to my AnnotatedString I have to monkey patch Camelizable#initialize.

No, it means that the implementation of Camelizable#camelize needs to be polymorphic w.r.t. my_string. Since Ruby is duck-typed, presumably the implementation of "camelize" would first check to see if the receiver responded to "gsub", then do the necessary substitutions. (Actually, a better implementation would be to first check if my_string responds to "camelize" and call that directly if it does.) In the end, this is a classic decorator pattern.

if you go the way of a decorator pattern, say, I write my `IntegerCamelDecorator` since your `Camelizable.new` does not support numbers, then what is the point of `Camelizable` to start with? You can just have a mirror hierarchy of `CamelAdapters`.

Of course, you can implement `Camelizable.new` so that it performs a dynamic dispatch itself, by looking up in a `CamelizableRegistry`. You can even have this built up magically with reflection.

You can, of course do everything, but a method call in ruby still only dispatches on self, and if that is fixed the method cannot be polymorphic, if you think otherwise we can agree to disagree.

You're focusing on the #new call. The actual #camelize call is still dispatching based on the Camelizable object which is parameterized with my_string. Ok, so it's not true call-site polymorphism...fine. I'd argue that's an unimportant implementation detail, but if you prefer (and what I frequently do in my own code) you could have something more like:

    Camelizable(my_string).camelize

where now Camelizable does the dynamic class lookup to choose the correct decorator for my_string. This turns #camelize into a truly call-site polymorphic call and, at least in my opinion, is far more readable/reasonable than either monkeypatching or refinements.

`module_eval` and `refinements` seem like good ideas unless you mix them. if `module_eval` wouldn't be so widely used op's arguments about what refinements do to code readability would mostly fall.

...why not simply just find way to discourage people from using these together, like having blocks of code only see refinements from their defining scope, not the one they are actually run in?(this would make it obvious for library authors that mixing these to features is "not sane to do", if the fact that they are used to solve similar when it comes to creating DSLs problems does not make it obvious)

Does anyone know how Objective-C categories handle some of the issues brought up with Ruby 2.0 refinements?

It seems method swizzling is a far less touched trick than monkey patching, and the latest LLVM compiler throws a warning if a category overrides an existing method. I am curious about low level differences in how the two languages approach this problem.

Objective-C categories are just like monkey-patching in Ruby. You can have collision with categories in Obj-C just like in Ruby, you'll just get a compiler warning (maybe error).

As of the latest Xcode linker, you also get a linker warning (which is more important in this context).

Method swizzling and refinements are basically the same thing. The real difference isn't in the features, it's in the fact that Ruby is a dynamic language, and Obj-C is only dynamically typed. You can swizzle all you want in a statically compiled program, and refinements work equally well in a "static" Ruby program. The problems with refinements start coming to light when you start refining things at runtime, i.e., with module_eval. Obj-C doesn't have this problem, since swizzling happens only once at compile time.

I don't believe that is true, you can use class_replaceMethod to dynamically swizzle methods at runtime. This gets rid of compiler error and warning messages that appear when just using categories to override existing methods. From my understanding, the compiler enforces compilation unit visibility for methods implemented in a category, but the methods actually are actually added to the global scope.

There is no "top level" object or class_eval in Objective-C, so the situation is quite different from Ruby.

Here is an interesting paper about classboxes[1] (same concept as refinements), and the source code for the Objective-C runtime[2]. I found them to be useful in my research.

[1] http://rmod.lille.inria.fr/archives/papers/Berg05a-CompLangE...

[2] http://opensource.apple.com/source/objc4/objc4-532.2/runtime...

> Obj-C is only dynamically typed

> swizzling happens only once at compile time.

Objective-C is a dynamic language in every sense of the term.

As examples, at runtime, you can:

- change an existing class' superclass - swizzle methods - create a class ex nihilo (though it'd be a better idea to inherit from NSObject) - give that class (or even an existing one) methods, ivars, and properties

.. and much more

Visit the runtime reference[1] and jump into a running Objective-C app with GDB or F-Script. It's a lot of fun.

[1] https://developer.apple.com/library/mac/#documentation/Cocoa...

Reply on news.ycombinator.com