Hacker News Reader

Comment by SoundAndBug | Hacker News Reader

What does solve the problem?

My take on it would be the approach from SICP[0]: separate data (objects) from transformations (streams). On a personal note I would also like to add:

- keep data serializable

- cluster transformations accordingly (Keep things that belong to each other as close as possible to each other, in a literal meaning, same file/module/etc. A new coworker should have the feeling of entering a public library - she'll know where to look for.)

- it will probably never be the case that you have to invent a new data structure or algorithm

Developers understand such terms like 'function' in very different ways (see top comment in this thread). Some have a more abstract approach and understand it as 'pure', where others have a more instruction-bundeling approach and understand it as 'procedure'. Either is valid, but are completely orthogonal to each other when it comes to its usage. I understand Gary Bernhards approach to 'functional core, imperative shell'[1] as a mean to talk about this - instructions demand for composable building blocks, and those building blocks come either from:

- built-in functions (with an already universally understood API)

- or your own (clustered) utils (and demand an easily understandable API).

[0]: https://mitpress.mit.edu/sites/default/files/sicp/index.html

[1]: https://www.destroyallsoftware.com/screencasts/catalog/funct...

I don't see how this solves the problem. I can have pure functions for parsing the HTML of each site: they take the text and return a simple data structure with the information I need. Chances are, if you write one and then another, you'll find there's plenty of duplicated code. Should you try to decompose those parts into common (pure) functions or not?

I assume your API can be expressed as this function signature:

  parseDataFromHTMLResource :: Resource -> HTMLRaw -> Maybe Data

...and assuming we validate the HTML:

  validateHTML :: HTMLRaw -> Maybe HTML

I think your question relies on the separation of 'Resource', as a domain separation like right below will probably create a lot of duplication (since any resource is free to structure their HTML within the standard however they want):

  parseDataFromGoogleHTML :: HTML -> Maybe Data
  parseDataFromGithubHTML :: HTML -> Maybe Data
  parseDataFrom...

Perhaps you can reduce the duplication by converging to different fingerprinted resources. So a HTML resource, fingerprinted by style, will then guarantee your data:

  data HTMLFingerprinted =
      HTMLStyleGoogle
    | HTMLStyleGithub
    | ...
  
  htmlFingerprintedFromHTML :: HTML -> Maybe HTMLFingerprinted -- So the Maybe will be here
  parseDataFromHTMLFingerprinted :: HTMLFingerprinted -> Data  -- ...not here

(Note that this may be what the parent means. It helps solving "For me it's usually, "Oh crap that thing I changed I had to change here and here too, whoops its good now... Wait no I also had to change it here... and here... now that we're done with that we should be fine... DAMMIT!"" with "I just create this abstraction".)

I would say if you're able to implement the function 'htmlFingerprintedFromHTML', you deserve the abstraction and thus DRY the codebase on the fly. And this "no pain no gain" mantra is what I personally really like on a 'functional core'. Code stays only duplicated where absolutely needed until you find a solution for the abstraction.

...and I guess implementations of fingerprinting HTML may vary a lot.

Yeah, I have no idea. Code that avoids this problem is mostly DRY if you look after the fact, but going from the DRY principle to code does very often lead to the wrong abstractions (that won't solve the problem at all) and too little repetition (that will make your code break on the opposite way: you change one piece and unrelated stuff breaks).

DRY is a good learning tool, and it is an after the fact property of good code. But people shouldn't ever preach it.

Reply on news.ycombinator.com