Responsible Monkeypatching in Ruby
blog.appsignal.com
blog.appsignal.com
1: https://ruby-doc.org/core-2.5.3/doc/syntax/refinements_rdoc....
I read somewhere recently that even for MRI ruby you can take a 40% hit when using refinements (but please take that with a big pinch of salt as I can't find the source right now).
0. http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/...
Note that this overhead apply as soon as a refinement is defined for a method regardless of wether it's ever active.
That 40% figure is for an empty method. So for "big" methods that are infrequently called it's probably fine, but it should really be avoided in hotspots.
For me refinements are a bit like meta-programming; I'm glad they exist but I think there's other ways to get the same result which are easier to live with.
Admittedly they're a bit of a mystery to lots of devs.
Writing good code is about clarity and working within maintained expectations. People reading your code should be able to know what it does. If someone joins your team and knows that Method X in library Y is bugged, they will expect it to be bugged in your codebase as well, and reasonably so. The fact that you monkeypatched it is not something that they will be aware of, and can potentially cause bugs in their code as a consequence, costing them time and patience.
A better approach is to use a wrapper instead of monkeypatching, or inherit from the bugged class and overwrite the bugged method. You get the same results, but anyone reading your code, with or without prior knowledge of how your code works, will:
1. Have immediate understanding that you've altered the source code in some way
2. Have a crumb trail leading them back to your custom implementation
Damn it, you got a language with namespaces, why do you think that is ?
Python is just as dynamic as Ruby in a lot of ways. But in Python, overwriting an existing and important/fundamental/ubiquitous method would be an absolute last resort when all other sensible approaches have failed. And in that case you would at least have the sense to save the original method somewhere under a different name.
What makes such a horrifying idea so apparently not-horrifying in Ruby? The ability to "reopen" class?
None of the examples in the blog post seemed like such dire circumstances as to require it. Maybe replacing a buggy method while you wait for a PR and new release is a valid use case. But the failure mode is also particularly nasty.
Since IBM went all services and consulting and practically only mainframe remained of the old IBM, and whilst x64 Etal went into architectural Cambrian Explosion, at the turn of the century, IBM Z division has become a entirely different animal. The just discussed at Hot Chips 33 Telum processor with CISC instructions for memory architecture networking and in pipeline RNNs is so out there that I have started to try and figure out how much it takes to get to be a IBM partner just to get access enough to write about this technology freak. If the team exists the finance is slam dunk link with a infrastructure fund. This mainframe AI can be trained with regular tools and functions with really low latency - it's on the die. Just don't be ageist and train vets with clearances (for LOB or just because it's the right thing to do - my partner founded a headhunter firm for veterans and placed the vast majority in IBs after 6 months of his programme - and age no issue to CEO signed undertakings, my partner really hit it out the park with his programme quality, quant covered from fp math to martingales but it had to shut down because the people guy who kept everything from rabbit holing to fulfil my partners fantasy hedge fund ambitions died of cancer and a tech partner is necessary to free me to pick up the relationships which are as open as the day my partner kicked those doors off their hinges with some outrageous leverage of 80s mainframe knowledge that pumped the flow desk of the financial infrastructure organisation who's CEO didn't blink when my partner made the FT trampling over every unwritten rule of headhunting that he never knew existed like everything else beyond his screens. This guy has just shy of 50 years success behaving like this what am I to know? I am still recovering from the unending cardiac arrests I had for no reason until I realised that some relationships really do transcend everything. The UK incidentally has a enormous pool of senior management talent created by the government firing all civil servants who qualified vocationally instead of university. Everybody was summarily dismissed. Just before the 1997 employment act was introduced. I have worked with the director of the largest aide agency in budgetary terms after the Gates foundation whose office was a dozen or so people. It now needs 600 apparently. The logic was partner agencies didn't manage fraud well enough for the British government liking. Plenty better than we do without any relationships with the same leaky institutions that now take pride in embarrassing the Brits with implicit sentimental support from the agencies we cut off.
My point is not a self promotion but the promotion of a architecture hardly anyone gets near enough to speak about for comparison with the front line of web development. This is a unmitigated loss for everyone all around.
All that boastfull background is intended to directly criticise the nature of this technology discussed here that's controversial enough to elicit numerous warnings in the comments immediately after publication.
Given the search rank of HN and the difficulty with finding human readable (non expert) commentary on highly technical issues like monkeypatching I don't see how the existence of this discussion doesn't de facto eliminate the techniques and even potentially the technology as a serious option for critical systems deployment.
Consider the weight a browsing executive will place on what's said here.
In the example just given the application is applying node specific handling changes to in progress production code in a webserver talking to banks running transaction systems.
Consistency of the www service and presumably the user interface critical to customers and likely to be the only way to access account information (giving up our company x.500 terminal probably not even after corporate death because companies have been bought from administration only for grandfathered services often enough I considered trading them for a living) despite how much people will pay for the capability.
Monkeypatching tells me that the errors aren't in your code because why introduce the trade-offs and risks of the technique for blowing up all of your most impressive customer references at the same time? I'm guessing hard nothing much has to be done to the error messages.
But it's so hard to blow up CICS (assuming you read the docs) potentially even lacking in-depth mainframe knowledge the opportunity to launch lightweight java environments and FFI to your Ruby runtime or make the call via CICS from a SPI error manager in javascript that because CICS you know is giving you actual state, or creating a message feed for individual subscription to have a extra confirmation that you didn't lose their messages, and running this under the banks sysadmins purview whilst giving you the same certainty your updates applied with point in time rollback to exact system clock and transaction stamp and in flight transaction state and zero sweat about any potential consequences looks pretty attractive to where I am seeing this.
If I have totally misunderstood this I'd like to beg a point for the scenario that I have described : namely one where you are responsible for the ephemeral bleeding edge of web UX and (in my hackneyed eyes) making a stateless best effort ad hoc text delivery protocol statefull reliable and infinitely scalable also a GUI thread manager and connections manager and anything else I omitted to boot, behind which is a great hulking mainframe nobody wants anyone to touch because the lack of adoption can't overcome human fear of being responsible for borking a IBM Z no matter how indoctrinated everyone is that Z can't fault assumptions of complacency obliterate reason and so you're hoodwinked (joined that club long ago) into not looking up how you can run webserver tech on CICS keeping it on the mainframe and getting the benefit of all that reliability and fault tolerance and data processing invincibility for your own personal gain.
Putting the code on CICS is slam dunk argument because how can anyone beat the mainframe management magnificence mantra unless there's some major issues with the big iron priesthood. Your code automatically gets the same management and reliability capabilities and gives sysops single pane of glass sight of everything happening with the front end interactions opportunity to get in on the new wave potentially for them personally and a nice brag of relevance in the face of the bleeding edge and unstructured world we represent, meanwhile the web development team just bagged major C Suite kudos and the ability to coopt some of that incredibly valuable institutional invincibility inventive that can transform your business model from job quotation to Senior Web Scale Z/OS Customer Systems Integrity Consultant times head count and paid pitch fees, retainer and expenses. Plus probably the possibility of getting insurance cover for DR scenarios and hourly overtime on any incidents paid for by customer DR budgets and policies instead of being the wrong side of pointing fingers and the one thing that C Suite types sympathise with is web UX and general web reliability grief.
I surely deserve criticism about my style with this, but what's going on with the mainframe about now is really wild. My pitches above were hypothetical to illustrate the point although my partner and good buddy is as much work as that non fiction description and a leading dinosaur in my eyes. I absolutely accept that the chances of mainframe ubiquity are not meaningful numbers. But with everyone busy making the OS as irrelevant as possible, how long does it take for the mainframe Telum cpu to be viable mainstream silicon? Very strangely I think I am going to live to see that move being made. Maybe the IBM strategy is the right one and only the attitude of management the problem.
For 'reasons', I need to add callback hooks into a particular HTTP Client library my org uses across several dozen of projects. The library we commonly use doesn't have hooks.
I've got a couple of options: switch wholesale to a different library, build a child class, build a nasty bit of monkey patching boiler plate, or build a little library which prepends some logic into the ancestors chain.
The first and second would cause eye-rolls ("I'm just calling an internal api in the way I've always done, why are you making me change it everywhere!"). The monkey patching boilerplate, well, gross. But a little bit of prepend logic packaged into a gem and I've got the behaviour I need.
One of the tremendous joys I get from ruby is never having the feeling that the language or runtime can't do something I need. Whether that thing should be done or which way it should be done is another matter.
The consequences are always: code unmaintainable over the long term.
Unfortunately to reach this disillusioned state you need to hit some really hard walls, which doesn't happen to all ruby devs.
So in doubt no metaprogramming. Example: I never wrote a macro in a real world Elixir project.
But it's not abnormal for a rails codebase to have a Concern (rails module) that provides a class method that does something, specific to the application. Which is equivalent to a macro.
> Unfortunately to reach this disillusioned state you need to hit some really hard walls
I have seen quite a few Rails apps, some pretty old, and these were not the hard walls which were hit. One way of not hitting those specific walls (which are pretty soft IMO) is hiring decent people, decently compensated. And empowering them to make decisions.
A lot of codebases do that, and they create chain of undeclared dependencies between multiple modules being included that make impossible to determine the requirements to use such module in the first place.
The development tend also to slow down to a cripple, as well as being overstaffed to compensate.
Yes, of course you can do horrific things with it. But the ruby community has largely been absolutely stellar about not doing that - most keep insanities tightly bounded, where they can be used to make pleasant and error-resistant APIs. And even when they're not (which is usually self-inflicted by internal code), the language is flexible enough to let you unfuck just about any fuckery that you run across, if needed.
But beyond that though, the vast majority of issues I've seen with monkey patching has been stuff that was created in the project/company that's experiencing the problems. Libraries are generally very good about how they monkeypatch (because doing it wrong with hundreds of using-projects VERY quickly runs into problems), but those ad-hoc internal monkeypatches are routinely done by people who don't fully understand what they're doing, or take shortcuts. Those can have latent bugs linger for a very long time, and yeah - they can be nasty to unravel.
Even this is becoming less and less necessary as more of the commonly-used syntactic sugar gets pulled into Ruby core.
static TimeSpan Weeks(this System.Int32 n)
{
return new TimeSpan(n*7, 0, 0, 0);
}
static DateTime FromNow(this TimeSpan dt)
{
return DateTime.Now.Add(dt);
}
static void Main(string[] args)
{
Console.WriteLine(2.Weeks().FromNow());
}I don't understand how monkey patching is beneficial beyond personal preference.
The way Ruby does metaprogramming makes debugging extremely frustrating because your stacktraces will contain references to methods that exist nowhere. Good luck then trying to find where they're defined so you can understand how they work.
I have a very strong opinion that generating methods at runtime is a terrible feature that should be used only when nothing else can be done.
At the top of any file:
def Weeks(arg) end
That's negligible.
The one I consider the most annoying: it substantially increases load time. ActiveSupport is huge and just requiring it increases the load time by 1.x seconds. Given what it provides, that's substantial.
The inability to extract a library (or a service) without depending on ActiveSupport is a cost that doesn't exist with the other approaches.
Not to underestimate, your _number_ now responds to `weeks`, and duck typing is the only way you have "interfaces" in Ruby. I can easily picture having a "weeks" or days attribute on some entity and a service object expecting that, now there are subtle bugs when passing accidentally the entity's _value_ instead of the entity itself.
Yeah, they are "minor" things, but the point is dying of thousands cuts.
The last damage is more subtle: devs, even less experienced devs, will feel authorized to monkey patch. In the private codebases I saw (multiple) this is common. It's not a last resort anymore and it ends up becoming a hard problem to debug and resolve.
The ability to monkeypatche pushes devs to NOT design their objects in an extensible fashion. This effectively is the highest damaging consequence, but it's incredibly hard to assess.
I'm not sure what you mean; Rails depends on ActiveSupport so if you're a Rails codebase you don't really care, you already depend on it. If you're a gem author it's your own decision if you want to depend on ActiveSupport or not, I'm sure many gems don't depend on it, and others do.
> Not to underestimate, your _number_ now responds to `weeks`, and duck typing is the only way you have "interfaces" in Ruby. I can easily picture having a "weeks" or days attribute on some entity and a service object expecting that, now there are subtle bugs when passing accidentally the entity's _value_ instead of the entity itself.
Never seen this. Not saying it can't happen (can u give an example though? not 100% sure what you mean) but I've never seen it.
> Yeah, they are "minor" things, but the point is dying of thousands cuts.
This is a bit of a hyperbole statement wouldn't you agree?
> The last damage is more subtle: devs, even less experienced devs, will feel authorized to monkey patch.
I think it happens way less than you think; usually in teams of inexperienced people who discover Ruby and metaprogramming for the first time. But the worst monkeypatching bug I've ever seen was actually done in javascript; a guy monkeypatched some jquery method and broke 20000 websites at once (we were a widget startup). Monkeypatching isn't a Ruby thing it's a dynamic languages thing. I don't see Ruby/Rails culture just encouraging people to monkey patch String or Object. The fact dhh and a very experienced team of devs did it, and tested it and perfected it for years, doesn't mean you should do it. This is pretty common knowledge now and I think 99% of Rubyists would agree.
> This is a bit of a hyperbole statement wouldn't you agree?
I'm not sure, I see many small problems that slowly leads to what the Ruby community is now. That being said, if you look at how many things ActiveSupport monkeypatches, "thousands cuts" it's not an understatement: https://github.com/rails/rails/tree/main/activesupport/lib/a...
Why is this bad? Makes total sense to me. Now both forms work and hopefully it eventually makes it into the language like often happens. But until it does, why not monkey patch obvious mistakes?
Array has `include?`, Regexp has `match?` and String has `match?` and `start_with?`.
This is consistent: there is no S.
Now when I'm in a codebase, depeneding on ActiveSupport and which parts of it are included, the consistency is now missing. Is it with S or without S?
And what methods DO have the S version and which one do not?
That's where the monkey patching comes in, usually.
But to be clear, I'm stating the facts that many software developer see in the Ruby world when you join an organization that's growing tremendously. As usual, if you fit perfectly in the Rails target and the Devise target, you won't encounter problems.
The load time of a single test is a recurring problem in the Rails world though, so ActiveSupport adds up.
Monekypatching doesn't lead to crash (it can, but that's not the point), it leads to: - Less extendable code - Less learning - Less maintainable code
Even if the damage is small, given how competitive is the world, why taking the chance for no benefit?
Well that's the crux of it here, you see it as damage/no benefit while others in the Rails world disagree. It's a difference of opinions that won't be resolved tonight by arguing on Hacker News. I did like your comment about naming consistency in Ruby though, learned something new.
Actually this is kind of the whole point of #present? - it's not just a truthiness check.
The only falsey values in Ruby are `false` and `nil`, but in the context of a Rails app you might want to treat things like "" and [] as "not present" (e.g., a user leaves a text field blank).
Once you're used to doing that kind of thing, people will feel the urge to take things to the next level, augmenting standard functionality and replacing the original functionality globally. Maybe you replace console.log() with a wrapper that includes a timestamp in the output, for instance. Not insane, and manageable for a smaller project that you have your head around.
It falls apart awfully quickly as it scales though. If behavior in your system diverges from standard because of a maze of monkeypatching, then you have to learn those quirks or there is a lot of frustration. And the possibility of spooky breaking changes in seemingly unrelated components becomes a big problem.
This is personal preference, but I find it utterly absurd to adorn numbers with these time dimensioning methods. Why not volume or length too? Should I be able to say 2.miles? Why would an integer be innately bound to time, as opposed to a unitless mathematical abstraction? Ruby folks like to add these gimmicks that make code “read like English” and I find it very cutesy, for lack of a better word. When I see stuff like this, I’m distracted by pondering what the hell the magic actually means. Can I say 6.from_now? Yes, actually, I can. Does it make any sense that what I call “from_now” on is technically a unitless number? No, not at all. It is a pretense that it’s a dimensioned number for the sake of cuteness.