Couldn't agree more on this. Magic is good if you understand the underlying stuff but if you don't and you are once offtrack you are lost. Maybe it's also a matter of taste. However the hype and traction DHH could build when RoR started was quite impressive.
The issue is when do each of us, as individuals and/or teams, really need to understand more. And perhaps more importantly, when does it make business sense to leverage magic to get to market faster, build cheaper, or find market fit before worrying about building the most robust thing possible.
Ruby's freewheeling philosophy however means that even when you are the author of the code some other code might have reached in and changed something out from under you. Not only do you not know there is another layer somewhere doing stuff that impacts you you can't even reliably follow a path to find that layer and debug it. As long as everyone does everything perfectly this is a wonderful world to live in. But the first time someone breaks the rules and impacts you and you lose a week or more unnecessarily you'll understand the distaste that ruby fosters in some people.
It's more about being able to discover what you need to know that it is possessing full knowledge of everything you need to know.
I'm not entirely sure how a filesystem works under the hood (let alone the physical media it's on), despite interacting heavily with one every day. A bug involving a hard disk would be incredibly opaque to debug, from my perspective. I suspect that you find it easy, because you've spent the time developing an understanding of the system.
> Not only do you not know there is another layer somewhere doing stuff that impacts you you can't even reliably follow a path to find that layer and debug it.
And this is where our roles are reversed! I would have little trouble following that path, all the way down into C if needed- because I've spent plenty of time understanding the system.
The problems you describe are problems anyone would have with an unfamiliar system, and it's not Ruby's fault.
Way to reduce what he's saying into the classic, "He can't be depressed! There's starving kids in Africa dying!". You've taken his /observations/, misplaced the usefulness of those observations, and then reduced them into some absurd argument that he's not actually making.
Accessibility to the unfamiliar access goes wonders and reduces the "magic" feel. If ruby doesn't have that sort of accessibility, then it IS Ruby's fault. If that's what Ruby actually wants, that's fine - but holy crap, it's this sort of mentality that completely fucks over usability.
I actually like Ruby, but hate RoR, because of a very lacking sense of control, unless you are very deep into it. You have absolutely zero feeling for what it OK and what's not OK to do with so much implicit magic behind the scenes. And that's debilitating.
My interaction with RoR can be boiled down to this exaggerated dialog:
- Here, write your code in this pattern and it will work.
- And if I change it like this?
- No it won't work.
- But why?
- It's an opinionated framework. It's the opinion of some Very Smart People(tm).
- But this is my work. Why can't I do it the way I want?
- Because you are dumb. And your dog is dumb. You haven't grokked it. You aren't a hacker. Neither is your dog. Where are your hoodie and Nike Airs?
I think the key difference is that you don't need to know how that works. I think there's a question of how leaky the abstraction is and how often you need to look behind the curtain.
There's probably a SE paper somewhere stating this formally, but I figure well done implementations on top of abstractions only need to reference one layer down. It might be okay to inspect the state the layer below for performance reasons, but no further.
That means that a well done Rails app might want to inquire about the state of HTTP, but shouldn't inquire about IP addresses. And it might want to know about indexes on its databases, but not disk caching policies or i/o schedulers.
So the case of "too much magic" is not to become used to the discomfort of not understanding how underlying abstractions work, nor is it to stop using them. Instead you want to choose dependencies with well documented abstractions, implemented such that you can inspect if you need to.
This has nothing to do with Rails specifically. I see plenty of coworkers who dislike Django's large set of features, as too much bloat, or too much magic they don't understand. Instead of reading documentation, and becoming expert in Django's abstractions, they spin up Flask apps, slowly reinventing the wheels, but maybe not quite as round.
You can even search for "rails typo in migration" and the first result is a stack overflow post suggesting and explaining how to roll back the migration so you can edit it
That's a little extreme, surely this must be hyperbole.
That said, I don't have a problem with frameworks that are larger. Hell, I'm a .Net dev by day myself and that certainly doesn't meet that requirement. I just don't find it elegant.
Deleted comment
How about electrons? Can you draw me a graph of the electrons currently residing in your computer case/laptop? Can you explain to me exactly how your PHP code will look in binary?
Sorry to be snippy, but you're making a pretty audacious claim there. I don't think any reasonable software engineer/developer can claim to fully understand every underlying principle.
I mean, you can have an abstract, high-level idea of how things work, but do you really think you know the intimate details of every single piece of hardware/software involved in writing your code?
it's the difference between being able to wire up a logic gate from transistors with a basic understanding of how they work and to be able to wire up a cpu, ram and some io from logic gates, being able to wire up a computer from cpu chips, ram chips and IO chips.
Even if you don't know the exact details of the latter in principle you understand that whole device that you just built.
So the claim is not so crazy as it might sound initially.
I think this is a false claim because.
1. Rails is easier to learn than most of the things mentioned above, so not magic
2. It's perfectly reasonable, and good, to build on top of and leverage things that you haven't taken the time to learn intimately.
No, that wasn't the claim.
> 1. Rails is easier to learn than most of the things mentioned above, so not magic
Ignoring the strawman component: that's a matter of opinion, for some those electronic bits and pieces are actually simpler to understand than software.
> 2. It's perfectly reasonable, and good, to build on top of and leverage things that you haven't taken the time to learn intimately.
Yes, absolutely. Just like we don't require everybody that uses the highways to be car mechanics or road construction workers. Society would grind to a very rapid halt if we had to have an intimate understanding of everything we use everyday.
I did a dual-major in CS and EE, and then an MSc in CS. While I'm a little fuzzy on the specific details of the current transistor geometries (14nm?! that's wild!), I do pretty much understand the entire stack upwards from there. I could design a fully-functional computer from the logic gate level, although it likely would be slow as hell and be exceptionally boring. It's not my strong point, but I understand enough about what's going on.
Honestly, most of my electronic design work these days is analog circuits slung onto the side of a microcontroller, with code written in C. I'll do schematic design and capture, and board layout without too much difficulty.
Going up from there, I've written my own (toy) operating system from scratch. I've written Linux kernel drivers for both off-the-shelf devices and for custom devices implemented in FPGAs. High-bandwidth/low-latency TCP servers written in C. My own (toy) programming language. I've written C extensions for PHP and Python, and have done pretty intense profiling work on them; this requires a pretty deep understanding of how the internals of the interpreter work (part of my MSc coursework). And I've done web apps in PHP, Python, Ruby, and Elixir.
The whole point of this is that I'm totally comfortable going as deep into the stack of abstractions as necessary to solve whatever problem I'm facing. When I started learning Rails, I was mystified by all of the "magic". I'd never even really used Ruby, let alone all the extra stuff Rails does. But I set up the emacs-bundler library (https://github.com/tobiassvn/bundler.el) and started diving in. I'd work through a tutorial, and every time I encountered something I didn't understand, I'd dive into the guts of Rails (bundle-open activerecord) and read the code until I understood it.
As an example of going a layer deeper, I once encountered a bug in a Python script that used PySerial. There were multiple threads, and the script would hang sometimes. I read through my code and couldn't find any problems, so I dug deeper into the PySerial code. Nothing obvious in the .py file for PySerial, so I dug deeper into the C extension. In the C code, I discovered that there was one case where the GIL wasn't being properly released before doing a (sometimes) blocking call! Tada! The script worked great if the call didn't block, but hung if it did.
I think we should all aspire to this level of having enough of a mental model for all the layers underneath us that we can go dig into them when needed. But it takes a lot of effort to get to the point you're at, so it's a pretty big ask. (Also, most of us probably can't go back to a university to learn the EE side, and it's not easy to self-learn it.)
Having said all that, for Rails specifically, it's pretty easy to do exactly what you did and just open up Rails and other library code. It's almost entirely simple method definitions, and the parts that aren't can be understood if you go read about how `define_method`, `method_missing`, and `const_get` work. The "Rails is magic" meme seems to either scare people off from reading the code, or give people permission to be lazy about it.
Install pry, pry-rails. Something weird occurs? binding.pry, `show-method` or `show-source` and it dumps you to the exact line and filepath of the method, be it the original or a monkeypatched method (use -a to show them). Keep digging like that and you'll eventually hit the C implementation code.
[18] pry(String):1> show-method length
From: string.c (C Method):
Owner: String
Visibility: public
Number of lines: 5
VALUE
rb_str_length(VALUE str)
{
return LONG2NUM(str_strlen(str, NULL));
}Sure, I can concede that code that tries to do too much for you to the point where you didn't actually want it is annoying. It's just that, in the context of what Ruby CAN look like, Rails isn't that arcane.
For example, entirely for fun (it was a lot like a game - I couldn't stop seeing how far I could push it), I gradually built up a small library (less than 100 lines of code) that eventually yet radically transformed Ruby into what looked like a badly designed yet different (functional) language. Functions could be conveniently composed, partially applied, juxtaposed, etc by monkey-patching the symbol class with overloaded operators. I also had fun getting all this working for instance method calls so they could be treated the same way (few caveats with that though).
Not to say that this is a production-worthy effort, it's simply not. It's just the experience that has ultimately taught me the extent of Ruby's (perhaps dangerous) power and that Rails doesn't actually call upon that much of it in the grand scheme of things.
Truly knowing Ruby for Ruby and what it's features can really do, you only need to look at the so-called magic that Rails demos to understand intuitively how it does what it does.
Another subtle thing with it is that, while I'm writing code in whatever scripting language I do think about what's going to be happening under the hood non-stop. For example, every time I write a line of activerecord, I'm thinking of what kind of SQL is likely to be emitted, and whether or not it'd even be possible for the database to have an index for the query. Sometimes a table scan is the best you can do without denormalizing and its awesome to notice that right away instead of down the road when it's slow.
But this all is both a blessing and a curse. Often it ends up meaning that I spend more time thinking about a problem than I need to.
Another fun example from some recent work: a client calls me up and has a machine that they can't access. More probing reveals that the machine is buried under a frozen highway! (I'd love to go into way more detail but... NDA) Anyway, we do still have an RS485 link to it, with getty running. The goal was to determine whether it was a software problem (fixable without needing a backhoe in 4 months when the earth thaws), a hardware problem, or a cabling problem. How do you know for sure? Jump into the bootloader and configure the ethernet phy to tell you what it sees by directly writing registers over MDIO! Turns out the PHY is alive and well, but can't seem to see the 100MBit sync pulses. Plus it is still successfully doing PoE negotiation. The cable is still connected but either the switch, the cable or the magnetics are bad. Field guy is going to try a new switch before we abandon all hope, but at least we know that no amount of software fixes will solve the problem