Programming with a love of the implicit
m.signalvnoise.com
m.signalvnoise.com
I may as well come with fresh eyes for any code I wrote over a year ago. I'm reading _my own code_ with an eye to figure out how it was put together. I should be courteous to my future self and to anyone else who wants to read my code.
I pick up a new framework every year or two. I've been a full time software professional for over 16 years and I'm picking up new tools all the time. I prefer tools that tell me things explicitly so that I don't have the steep learning cliff of grokking the entire system in order to be able to work with one little part.
And isn't this hypocritical? So many people (probably DHH too?) gush about how easy Rails is to get started with, but here DHH argues that it's so much butter for the veteran who knows what all the implied magic is. Doesn't all that implied magic also leave the beginner kinda helpless? There's so much Rails tutorial with the attitude "here, do this, trust me, it'll work". And I've found that trust to be misplaced. The underlying implied design isn't actually what I wanted, and it's really hard to get it to do what I want because the convention is so baked in.
On the other hand, Go is a bit tediously boilerplate heavy. I'm still looking for the right balance.
the analogy I'd use is driving a car versus sitting in a passenger seat of a car. When I'm a passenger, I never remember the route we took. I retain almost no information about lefts and rights and landmarks. But if I'm driving, I really only have to drive to a place once to remember the way.
As someone who got 'started' (to an extent) with Rails, I both agree and disagree with this paragraph.
The main benefit of Rails, for me, was that once I dove in and try to understand how things worked, I learned a tremendous amount. Contrast that with my situation now, years later, where I'm piecing together 'best practices' for React/Electron/Cordova apps from various articles and tutorials, and I can conclude Rails really helped me along.
On the other hand I do agree that the tutorials that you mention weren't much help, if not downright harmful in my journey.
All that said, I currently lean towards the side of explicitness for actual work, and Rails goes a bit too far for my tastes. But then I'm bit of a control freak.
If we just go down another level (where you would implement something like Ruby on Rails), the same question appears. Do we favor explicit logic over implicit relations? At that level, I'd say the rule of thumb that "explicit is better than implicit" pays off, because it's often not that much more code, and a lot of code builds on top.
On baked-in conventions, the implied design of a thoughtfully designed framework likely has wisdom that you shouldn't too quickly dismiss, and should at least fully wrap your brain around (along with the complementary design decisions in the rest of the framework) before diverging. The most baked-in assumptions in a well-designed system are the ones that exist for the strongest reasons.
Take ASP.NET, for example using method attributes for authorisation:
[Authorize(Policy = "AdminOnly")]
public IActionResult Method(int projectId)
{
return Ok();
}
To actually build this without a clumsy if/else and manually returning 403 with JSON (to describe the error) in every controller method, it was necessary to create an exception and a custom exception handler middleware, and in the handler, produce a non-standard HTTP code and catch it again in a second middleware after the framework would otherwise gobble up 403s and return HTML instead of JSON. Exceptions inside controller methods are a perfectly adequate model for authorisation failures, but because the framework expects you to hook into their middleware for it, they made it very difficult to do it any other way.
Ok, that was a lengthy example. But it illustrates what I mean by wanting to write my own magic. Frameworks shouldn't cripple userspace magic by design.
Lisp let's me understand how procedures go together to build things. In particular, it helps get to a dual phase of programs. Macros aren't my program running. Rather, they are a program that makes the program I want to run. It just so happens that being able to cycle between those phases is very useful.
I can make no claims that one is superior.
Edit: s/string/strong/.
Jet.com has an interesting post on why they chose F# for similar reasons [1]
[0] I'm working through the How to Design Programs book from Matthias Felleisen which I highly recommend despite its rough edges http://www.ccs.neu.edu/home/matthias/HtDP2e/
F# seems to be getting more attention, but I mostly just see people struggling to get EF migrations going, and it seems general ecosystem support is still shoddy (compiler bugs on .NET core 1.1? 3rd party VSCode support? official VS plugins crash all the time?). I have to use C# for work, so it's just not stable enough despite F# itself being around for ages :)
I'd definitely like to give it a go for some bigger projects.
> On the other hand, Go is a bit tediously boilerplate heavy. I'm still looking for the right balance.
I've been working with Django (in Python) for a while now, and find it strikes a really nice balance. Almost nothing happens by magic, but there's also surprisingly little boilerplate. It used to be a little rough, but it's gotten a lot better in recent years especially (I think likely due in no small part to influence/competition from rails), so if you haven't checked it out recently, you should!
Power without magic.
Ruby itself is strange (in implicit way) enough on how things are defined and called. Maybe it's just me, I don't even understand its OOP although I came from C++ background.
That's irrelevant to his argument though, as he doesn't speak of coming with fresh eyes to the code, but coming with fresh eyes to the framework/language.
Even if you have forgotten why you wrote some particular code, you'd still remember the ways of the framework/language from other code you've since written with them, or generally because they're more general and fundamental than your particular code (the same way you'd remember the language's syntax, etc, but might have forgotten why you wrote some algorithm a particular way).
I think the trick is to come up with the right abstractions. I think we need to strive for abstractions that are relatively easy to grok (at least as a user of the abstractions), such that it doesn't feel like magic as much as it feels like a well designed language for the problem domain.
Of course, that's easier said than done. But most things worth doing are hard :)
Various strongly-typed languages have something like that, with tooling that can add and remove explicit types from variable declarations and generic method calls, make calls of extension methods explicit, add explicit arguments for functions taking default values, etc.
That can give you longer code when you don't know the language and/or the program at hand well, and terse code when you do.
Go is a half baked DSL gone too far. Just like PHP, in 15 years, people will regret that the language wasn't carefully crafted despite its good "ideas", when developers will have to maintain million of lines go projects in production.
When code is written only once and most of the cost comes afterward, that seems like an impossible choice to defend.
Why optimize for keystrokes? How much do keystrokes really cost you? Why optimize for conciseness, if it comes at the expense of readability?
I've maintained some PHP code bases with tons of implicitness (magic methods in ORMs and strings dictating object behavior, mostly) and some that had no implicitness. Guess which were more expensive for the client and painful for me?
Another example: I've recently been working on a C#/.NET code base, and I was wondering where the HTTP request is converted into a C# object (in this case, a JSON to object deserialization).
Guess where I had to go to figure it out? Google. And it wasn't easy to Google. I couldn't follow the code because of the layers of magic and configuration that controlled my code, rather than actual code. It was a huge waste of time and makes the code base completely opaque sometimes. It's a ball of yarn, and you don't know where it begins or where it ends.
Why is my code behaving strangely? Is it because I wrote it to do something a certain way? No, it's because this random config file (that I didn't know existed) is dictating behavior instead of the code itself.
I can't imagine how anyone working on large code bases with other people would want to do this. Yes, implicitness is more fun and beautiful at the beginning, but it becomes a nightmare after a short time for anyone other than the original coder.
> I can't imagine how anyone working on large code bases with other people would want to do this. Yes, implicitness is more fun and beautiful at the beginning, but it becomes a nightmare after a short time for anyone other than the original coder.
Good points, it seems like the arguments for implicitness may have been stronger in the past, when programming languages were less developed. Think of RollerCoaster Tycoon being written almost entirely in assembly in '99 by a single programmer. You'd have plenty of incentive for implicit standards. When you have modern languages with well optimized abstractions, all that implicitness ends up losing out. But if you compare the total amount of work that went into, say, Rust, with the cost of a single dude just building an awesome game, you see that explicitness only wins when it gets to cheat and use way more resources. So yes, explicitness is always better in the limit, but when resources are more constrained, implicitness is so nimble it will just crush the competition.
Are you saying that Assembly is more implicit than Rust or C++?
What do you use to measure "implicitness"?
> > Think of RollerCoaster Tycoon being written almost entirely in assembly in '99 by a single programmer. You'd have plenty of incentive for implicit standards
On the other hand you could say that it's the rust compiler source that is rather explicit about the mechanics and the game code would be explicit only by extension.
I feel that this argument has been repeated so often that it's almost understood axiomatically: nobody thinks about what it means anymore, it just "seems true".
I posted Aaron Turon's (of the Rust core team) article that touches on these topics elsewhere in the thread (https://news.ycombinator.com/item?id=14280908), but this topic desperately needs a better definition of "implicitness".
Virtually nobody thinks that C programming is "too implicit" because it's "optimizing for the writer over the reader," and similarly, garbage collection is an extremely implicit mechanism that is widely accepted, including by some of the strongest EIBTI proponents.
On the other hand, as a strong proponent of abstraction and implicitness in many contexts, I was extremely supportive of (and helped champion) Rust's explicit error handling through the Result type.
I like Aaron's "reasoning footprint" rubric because it gives us a way to debate this topic without "implicit" defined in the eye of the beholder. Importantly, it allows us, as a community, to broadly accept changes like garbage collection and control-flow constructs without repeatedly rehashing the same old bumper-sticker debates that mar every generation of programming.
I think about what it means constantly because every single day, I spend hours digging through documentation about the .NET framework instead of just following a call stack or reading code. It's excruciating.
Do I know exactly where to draw a hard, absolute line between "too implicit" and "too explicit"? No. But I definitely feel the pain of "too implicit" when it's happening to me.
> I feel that this argument has been repeated so often that it's almost understood axiomatically: nobody thinks about what it means anymore, it just "seems true".
Indeed. It seems just another way to not understand "everything is hard to read until one learns to read it".
So everything is equally difficult to read and understand? There are articles that say "great software is like Shakespeare" and others that say "great software is like Hemmingway". Which one of those authors is easier to read and understand, as someone who speaks English natively?
As someone who has inherited just two code bases, I know first-hand that there can be massive differences in how easy something is to read and understand. One of my companies had a code base that non-technical people could read and even send PRs for. Another of the code bases took months of ramp-up time.
> But this, in my opinion, is a misdiagnosis of the problem, one that throws out the baby with the bathwater. The root issue is instead: how much information do you need to confidently understand what a particular line of code is doing, and how hard is that information to find? Let’s call this the reasoning footprint for a piece of code. The pitfalls above come from the reasoning footprint getting out of hand, rather than implicitness per se.
> Does readability then demand that we minimize the reasoning footprint? I don’t think so: make it too small, and code becomes hopelessly verbose, making it difficult to read by forcing too much information into view at all times. What we want is a sweet spot, where routine or easy to find details can be left out, but relevant or surprising information is kept front and center.
[1]: https://blog.rust-lang.org/2017/03/02/lang-ergonomics.html
No one would say that if you have an almost-perfect abstraction, like a physics equation for fluid dynamics, that you should go to the effort of dropping down to the lower level (specifying what each molecule should do, in this case).
The problem is that this ORM example is an extremely leaky abstraction. Several of the inferences it is making are inferences that are not predictable if you don't already know how it will work. And they are inferences that matter.
Anyone who has ever used an ORM knows that it seems like magic, right up until you try to run some large query and it takes an eternity, and then you have to unpack it all and add all the explicit stuff to fix the problem that you "avoided" in the first place.
The real art of programming is in knowing what needs to be explicit and what can be implicit. I'd agree that if something can be made implicit without mystifying callers, it should be.
You rarely have to "unpack everything", you usually just need to go down a level of abstraction or two to get at the actual database driver in specific parts of code.
The trade off of using an ORM is still almost always worth it in my experience, but it certainly depends on what your app does.
schema "people" do
field :first, :string, null: false
field :last, :string, null: false
belongs_to: :company, App.Company
has_many: :assignments, App.Assignment, foreign_key: :person_id
end
def create_changeset(attrs) do
%Person{}
|> Ecto.Changeset.cast(attrs, [:first, :last, :company_id])
|> Ecto.Changeset.validate_required([:first, :last, :company_id])
|> Ecto.Changeset.foreign_key_constraint(:company_id)
end
I think the win readability wins later FAR outweigh the few validations I have to explicitly type out. A new person or someone under a deadline can quickly open one of the Ecto schemas, see all the relationships, see the validation being done, and see why stuff fails. And by turning it into a function you must explicitly call, it makes the data the most important thing and not the ORM itself. I could have called that function "parent_changeset" or "lawyer_changeset" if those made sense for what those functions did; they're not specially named. And then when you get that, you still need to actually apply the changes to have any effect.
It's also extremely helpful when reading and debugging others' code bases. Having to construct systems of rules in your head sucks. Being able to follow the function calls and data around while validating assumptions with the REPL makes figuring external bugs out much easier.
- Implicit code increases the barrier of entry to new coders. This is counterintuitive as there is less overall code to learn. But implicit code often relies on auto-magic. Until you know the pieces of that magic, you are left confused. Case in point, as a web coder to find the file for `render user`.
- Implicit code works great until you need something to work differently than the implicit nature intends. This is because implicit code is also opinionated code. When your feature needs to fight against those opinions, you can expect to write a lot of highly explicit, ugly code to break that mold.
The slider of explicit-implicit will be talked about as long as code exists. Implicit systems result in less LoC, but you risk sacrificing expressiveness. For growing applications supported by growing teams, some of that time gain can slowly fade as your team gets younger and less experienced.
The implicit code requires you check all of the implicit assumptions. Explicit code makes checking such assumptions much easier.
Now, raise your hand if you honestly think that the answer is "embrace implicitness" or "embrace explicitness" full stop. In that context, it's worth noting that the Tao of Python was written well after the language was constructed, as a summary of the choices that were made. It was not originally intended to be a design document.
Sometimes implicit behavior is easy to understand, predictable, and saves lots of time. In those cases, it's good. In others, it borders on madness: http://stackoverflow.com/questions/12488705/incorrect-plural...
What determines the cases where one side of the tradeoff is good, and the other is bad? Show me cases where Rails does better than Django or some other framework and vice versa, and tell a story about how you identify those cases and judge whether the tradeoffs are worth it. That would be an article worth studying.
I don't like over-explicit code like Go forces on you, but if I was forced to choose between over-explicit and over-implicit, I wouldn't have to think twice about going for the explicit language.
Explicit code is great for business level logic. In any business, and especially startups, things are changing so frequently that there's often very little in the code that could be set up as a convention and the accompanying "magic" that has to make it happen. Sales processes change weekly. Frontend features change daily. Reporting needs can change hourly.
In those environments you have to be able to quickly look at the code and see/know exactly what it is doing now and why so that you can be better able to make changes without introducing defects. Testing helps this a lot, but succinct, readable, explicit code makes life so much easier.
Implicit for frameworks; Explicit for business.
Plural table names are already pretty stupid, but using "people" vs "persons" even more so.
Implicitly having the table name match the entity after some case transformation is fine, but it should be more direct and not require an English dictionary to handle edge cases.
http://www.dylanbeattie.net/2017/04/theres-problem-with-phal...
Rails lets you write code that can be read out loud and still make sense. You can have a meeting about an ActiveRecord class and not get caught up clarifying what you're talking about every step of the way. You could put some code in a presentation to your manager if you really wanted. In three years' time when you look at your code, you can read it aloud again and find yourself saying exactly what it does. Rails lets you describe ideas at a similar level of abstraction as common English speech. That has value, even if it lacks precision. If having a dictionary embedded into the macros is what helps people write code well, then they clearly value matching up with their brain over matching up with a compiler, and good for them.
If I'm read app code, a "person" is a variable and a "Person" is a class.
If I'm reading SQL a "person" is a relation (e.g. a table).
I consider losing structure and adding complexity just to be able to read something out loud a net loss.
So would I. But Matz wrote early on that 'Ruby is designed to make programmers happy,' and that is behind lots of language decisions in Ruby and Rails. Reading aloud isn't an isolated end goal but more a way of describing that design principle. Writing Ruby should be a pleasant interaction, and Ruby programmers should feel like they are talking to a friendly machine. There are many ways to write the same code, and you can develop a distinctive writing style if you want. Many Rubyists talk about and write code like it's poetry, and feel like they have a voice through the code they write. Can you say that about a single Java programmer?
This isn't an accident, and effects like this are not frivolous entertainment the Real Programmers can ignore. It isn't a perfect vision, and you're absolutely right that sometimes it introduces complexity, but it's a real philosophy with tangible benefits, and pluralisation is a tiny part of that vision.
class Person < ApplicationRecord
belongs_to :company
has_many :assignments
end
For every implicit thing in the code the programmer must have a mental model of what it is doing behind the scenes.
So one taxes the attention system of the programmer, and the other taxes her memory system.
Other things are both explicit and require a complex mental representation of state (Haskell), but the type system makes many classes of error impossible.
I think most people would agree with DHH's aesthetic when it comes to abstraction. What makes ActiveRecord's approach to abstraction get ugly in many cases is its coupling to the mutable state of the underlying database, which is not guaranteed to be in sync with the logic implied by the declarations, which may also have unforeseen timing side-effects.
Imagine the more typical scenario:
class BusinessContract extends ActiveRecord::Base
belongs_to :company
belongs_to :client
belongs_to :sales_rep
acts_as_attachment
acts_as_state_machine
acts_as_versioned
before_save :notify_rep
after_save :update_salesforce_api
after_update :generate_pdf
end
Decisions like this are, in my case, heavily influenced by how much I trust the rest of the codebase. Having "magic" code on top of a rock solid abstraction feels nice and safe. When I debug / troubleshoot I don't feel the need to step into the function to look at its implementation one more time.
I am much more likely to be explicit, suck it up and write the boilerplate code once again in a messy codebase where the abstractions change all the time (whether I'm the one who wrote the sucky code or not). In the end, I feel like it comes down to how well your objects / concerns are designed. When the concerns are properly separated and your objects are solid, magic code is indeed beautiful and extremely expressive.
I think it is often the case where one can argue that imperative programming is explicit, but in reality declarative syntax patterns are often easier to approach as a beginner and an expert.
The descriptions for an incredible number of projects on GitHub lead off with something along the lines of "$PROJECT is a simple ...". My theory is that preemptively framing one's projects as "simple" might be a subconscious defence against accusations of triviality.
To go a bit further and just state that suddenly it's NOT OK at the language level is kinda argumentless in my humble opinion.
And yes, there are degrees of acceptability. If JPA had an annotation like @OMGWTFBBQ and ActiveRecord supported something similar - it would be rejected. JPA and ActiveRecord do things like this, but much of the rest of these tools features outweigh their faults. I know for sure there are many Java frameworks that actually love it when you write your own annotation.
For example I use a @NoAuthentication annotation in Jax-RS. There is no standard annotation for that, yet in our company we all use it and know what it means. We mark this REST service as a service that anonymous users or authenticated users can call alike. This is because we favor a model of placing our authentication code inside the stack (in our case CXF) rather than adding code at EACH rest method. It's a very VERY implicit style, we can't look at and adjust our authentication model, but just the same, we SHOULDN'T be trusted to change our authentication model in each method.
Annotations are not exactly the same as implicits in other languages, but it's the closest example of something in Java that makes our lives much easier and hides a LOT of explicit info.
The more I learn, the more it seems an immense portion of high caliber software engineering is knowing how to approach and analyze very general questions related to abstractions, such as:
1. Given the current and expected future constraints, is this the best abstraction we can come up with?
2. Is the abstraction worth its cost? How heavily should we invest in using it, given how confident (or not) we are currently that it solves our problem well?
I think of abstractions as being everything from breaking an if condition into several local variables to splitting a large monolithic app into independent services. We could go further of course, but I think that gets the idea across.
On the other end of the spectrum are things like Scala's implicit parameters, which I abhor.
https://medium.com/@shazow/code-boilerplate-is-it-always-bad...
hope scala (or intellij?) would get a "find all implicits related to this file/folder" and other things to help me around
One problem with Rails is that these abstractions are applied implicitly and burried deep in the framework. For DHH it's easy to know what it's doing behind the scene. Not for people who did not create these implicitly applied abstractions.
So.. Either you will have to learn ALL the Rails conventions, or you can create your own abstractions and use them explicitly. Also, it will be easy to put a breakpoint on the call site to your abstraction. Try that with Rails, you don't know what/where to put that breakpoint.
class Person(models.Model):
company = models.ForeignKey('company', related_name='people')
Side note: the author has a short paragraph about "simple/easy" and "complicated/hard" products, which doesn't ring true with me. I would think most people would say their software product is "complicated, but easy to use".
This is the tax we pay for being part of the DHH cult following (some against our will) :p
Program with Ruby long enough and it really starts to feel like a religious experience. You feel like you can literally do anything. Things you can't do with it just don't seem important anymore.
Rails just takes Ruby and puts it on steroids.
Bad joke aside, and I may be a naïve newbie, but what is stopping someone from writing tons of implicit code during their most productive work hours and when they're tired go back and make comments that make the code more explicit. Seems that this would get the best of both worlds in that you'd be able to be hyper productive when you've got the mental resources and then taking that extra time to make sure whoever stumbles upon your code later doesn't have to deal with a huge learning curve.