Hacker News Reader

Comment by beaker52 | Hacker News Reader

I find DRY fascinating.

It's the source of a large portion of the accidental complexity I find in code. "If I just create this abstraction, all this duplicated code goes away" - we've all heard it and many of us have told it, but few of us realise that it's the prequel to the most popular story of all: "all this code is such a mess, there are all these extra layers that don't really make sense and unpicking it is such a pain, I can't believe someone wrote this".

The story inbetween is about a young, inexperienced developer who has 3-days to deliver the one-feature-to-rule-them-all, to appease the almighty project manager, necessitating an adventure into the labyrinth carefully crafted by the developer in the first story.

View on HN

DRY increases "connections" in the code (often called "coupling"). Pay attention to what you are connecting.

The dominant example of this in my mind comes from some traumatic (and dramatic) work experiences involving web scrapers.

You scrape pages A and B. Both require logins. You notice that A and B use similar code to login, so you factor it out and now you have "connected" A and B. Or, more accurately, the common "login" method is connected to A and B.

The problem is that A and B are separate ships, and they are going to different places, and you tied a rope between them. That rope is going stretch and fray and break. There's no reason logging in to A and B should be similar, it just happened to be that way at one point it time. One day, probably, those two sites change their login workflows to be completely different.

So "just repeat yourself" and let each scraper be self contained. Let each ship sail their own way, don't connect them.

One case of this isn't bad, but if your not careful you end up creating dozens of connections between things going all different directions. Break your project into "things" and graph their dependencies. If you can break X by changing Y, then X depends on Y. If you can break Y by changing X, then Y depends on X. Your dependency graph should look more like a tree than a total graph.

If things needs to evolve to separate direction, you can duplicate them.

Unreasonable X is unreasonable, that's true no matter X but that's not very insightful. Unreasonable factorizing is unreasonable. So are unreasonable copy pasts.

Now a good question can be: would I "prefer" one or the other gone too far? That's merely a personal preference question, and on my side I've got a clear answer: I prefer something a little bit too abstract, to something a little bit too copy pasted. Because while abstract I still can understand more of the system more quickly, whereas duplicated I basically have to start by reading a lot more and factorizing (possibly in my head) before getting to a balanced situation...

Your own preference may vary, but I've yet to see a system maintained correctly by people copy pasting too much and not caring about anything but the one example they have to handle (because e.g. of a workflow described in a ticket).

Things can be accidentally similar, but WAY more often in copy pasted code-bases they are accidentally diverging, and that multiply the time I spend on that kind of mess by a factor that is probably near 10.

Problem with abstractions is, once they exist, people assume they are meaningful.

That is because proving that it isn't is a major task that involves dissecting the codebase, its current specs and planned future features (a seemingly pointless abstraction now can be the groundwork for a new module that will appear soon[0]). Thus the asymmetry between adding and removing abstractions.

[0] where soon either means really soon because product management needs it yesterday or never because there is too much work to do in other places.

That's right. DRY, at the limit, is compression; in some ways, it's maximally coupled, such that small changes have large effects. The quality of the abstraction is what is levered up.

naively I'd think you can simplify them by having a single interface but with or without sharing internals (as you see fit)

but in a way that just pushes the problem to coupled interface signatures..

now I wonder, is there research in variability based interface design ? looking at systems and estimating how much some parts can change and how much are probably easy to pin down forever ?

A mutable variable causes even more "connections" as every function that touches that variable is tied to it.

If you construct your program via the point free style using function composition, the dependent function can easily be swapped out of the composition and replaced with the modified function that you need while maintaining DRY to the maximum possible effect.

You are talking about a fundamental question of program design that is largely solved by functional programming via the point free style.

It is talked about here:

https://www.quora.com/Is-senior-full-stack-engineer-Ilya-Suz...

> A mute-able variable

This would be a variable that can be muted. More commonly, when people refer to "mutable" variables, they mean variables that can be mutated. It's not clear what muting a variable would mean.

Sometimes a human makes a spelling error by accidentally placing a hyphen in the middle of a word.

From context, hopefully other humans will recognize the slight error and comment on the topic rather than walk away totally befuddled and comment only on the error.

Do you not understand what I am saying in the comment? It's too late for me to edit the comment, but I hope the meaning is clear despite the slight error.

I've fixed the typo ('mute-able' -> 'mutable') in your comment above.

Please don't be unnecessarily pedantic on HN. This is covered by the site guideline:

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize."

https://news.ycombinator.com/newsguidelines.html

Surprised no one linked Simple Made Easy[0] yet.

Around 30 min, Rich Hickey describes how the opposite of simple is complex, and mentions the etymology, "to braid together".

"It's bad. Don't do it."

0. https://www.infoq.com/presentations/Simple-Made-Easy/

Of course, but it could also be said that we're just composing the scrapers with a common building block, which Hickey says it's good.

As a counter, you have 10 scrapers with the same login code. You find one site has issues when they deploy occasionally and go and add retry logic to it, then next week find the same on another site. You go and copy paste it over all 10 or leave it. You find later the retry causes rate limiting problems so you again have to fix it in 10 places, or maybe someone fixes it in one and not the others because the implementations have drifted despite solving the same problem.

Or, you use the same code for all and if the sites change the login flow then you add a custom one for the site which is different.

Fundamentally, I don't agree with the argument that I should copy and paste code now to avoid the possibility of copying and pasting the same code later.

It would seem like retry is working at a different level of abstraction than logging in and that code should not live together. As such, there should be the opportunity to inject a role player that understands what it means to retry.

The base case of retrying is `zero retries` (or `one try`).

> As such, there should be the opportunity to inject a role player that understands what it means to retry.

In each of the 10 scrapers?

If it's common code for the retry, now all my login code needs to raise the same type of exception or otherwise signal in the same way that the login has failed. Except instead of the abstracted retry code needing to work with one login function it needs to work with ten independently maintained functions that happen to be the same currently.

Retrying and stuff like that could either be handled with inversion-of-control which requires building a set of interfaces that provide stuff like 'login()' which is made available to some kind of orchestrator that handles retrying and other metalogic. This requires understanding some new domain specific abstraction really well or providing ample "escape hatches" in the even the abstraction doesn't work for all cases

Or, in my opinion, handling it in the typical way functional programming does, you'd have your stateful computations like login represented as functions returning IO, which you can easily use off the shelf functionality to rate limit and retry (like cats effect, fs2, etc... In scala). This kind of programming isn't as mainstream as it could be, but if you can build retry once and use it for pretty much any side effecting computation, you wouldn't feel a need to DRY up things that should be separate in an attempt to share code.

To be fair, it is pretty easy to fork a shared component when necessary, but it is painful to maintain multiple components with duplicate logic. So it is reasonable to err on the side of DRY when in doubt.

It's the source of a large portion of the accidental complexity I find in code. "If I just create this abstraction, all this duplicated code goes away"

For me it's usually, "Oh crap that thing I changed I had to change here and here too, whoops its good now... Wait no I also had to change it here... and here... now that we're done with that we should be fine... DAMMIT!"

The problem is that DRY is not the thing that solves this problem.

It looks like it is, you'll see plenty of people claiming it is, it may even lead people into solving the problem after they gather some experience. But it's not what solves the problem.

What does solve the problem?

My take on it would be the approach from SICP[0]: separate data (objects) from transformations (streams). On a personal note I would also like to add:

- keep data serializable

- cluster transformations accordingly (Keep things that belong to each other as close as possible to each other, in a literal meaning, same file/module/etc. A new coworker should have the feeling of entering a public library - she'll know where to look for.)

- it will probably never be the case that you have to invent a new data structure or algorithm

Developers understand such terms like 'function' in very different ways (see top comment in this thread). Some have a more abstract approach and understand it as 'pure', where others have a more instruction-bundeling approach and understand it as 'procedure'. Either is valid, but are completely orthogonal to each other when it comes to its usage. I understand Gary Bernhards approach to 'functional core, imperative shell'[1] as a mean to talk about this - instructions demand for composable building blocks, and those building blocks come either from:

- built-in functions (with an already universally understood API)

- or your own (clustered) utils (and demand an easily understandable API).

[0]: https://mitpress.mit.edu/sites/default/files/sicp/index.html

[1]: https://www.destroyallsoftware.com/screencasts/catalog/funct...

I don't see how this solves the problem. I can have pure functions for parsing the HTML of each site: they take the text and return a simple data structure with the information I need. Chances are, if you write one and then another, you'll find there's plenty of duplicated code. Should you try to decompose those parts into common (pure) functions or not?

I assume your API can be expressed as this function signature:

  parseDataFromHTMLResource :: Resource -> HTMLRaw -> Maybe Data

...and assuming we validate the HTML:

  validateHTML :: HTMLRaw -> Maybe HTML

I think your question relies on the separation of 'Resource', as a domain separation like right below will probably create a lot of duplication (since any resource is free to structure their HTML within the standard however they want):

  parseDataFromGoogleHTML :: HTML -> Maybe Data
  parseDataFromGithubHTML :: HTML -> Maybe Data
  parseDataFrom...

Perhaps you can reduce the duplication by converging to different fingerprinted resources. So a HTML resource, fingerprinted by style, will then guarantee your data:

  data HTMLFingerprinted =
      HTMLStyleGoogle
    | HTMLStyleGithub
    | ...
  
  htmlFingerprintedFromHTML :: HTML -> Maybe HTMLFingerprinted -- So the Maybe will be here
  parseDataFromHTMLFingerprinted :: HTMLFingerprinted -> Data  -- ...not here

(Note that this may be what the parent means. It helps solving "For me it's usually, "Oh crap that thing I changed I had to change here and here too, whoops its good now... Wait no I also had to change it here... and here... now that we're done with that we should be fine... DAMMIT!"" with "I just create this abstraction".)

I would say if you're able to implement the function 'htmlFingerprintedFromHTML', you deserve the abstraction and thus DRY the codebase on the fly. And this "no pain no gain" mantra is what I personally really like on a 'functional core'. Code stays only duplicated where absolutely needed until you find a solution for the abstraction.

...and I guess implementations of fingerprinting HTML may vary a lot.

Yeah, I have no idea. Code that avoids this problem is mostly DRY if you look after the fact, but going from the DRY principle to code does very often lead to the wrong abstractions (that won't solve the problem at all) and too little repetition (that will make your code break on the opposite way: you change one piece and unrelated stuff breaks).

DRY is a good learning tool, and it is an after the fact property of good code. But people shouldn't ever preach it.

A lot of principles like DRY (as described out of correct context in this article) have cult like followings people follow mindlessly leading to unnecessary introduced complexity.

I'm always amazed at how eager people are to over-engineer a solution that makes it a mess to deal with moving forward. Developers at large like to appear clever, tend to have (fragile) large egos, and don't seem to want to veer from established dogma--much of which based on little evidence or evidence that doesnt apply to a case they're dealing with.

Reply on news.ycombinator.com