Hacker News Reader

Comment by Klathmon | Hacker News Reader

I love this! It's the "jazz music" of software development. Something which breaks all the "rules" but does so purposefully and explicitly so that it can become better than the "rules" allow.

A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements, has a lot of "pointless comments" that just describe what the line or few lines around it is doing, and has a lot of "logic" in the comments which could quickly become outdated or wrong compared to the actual code.

Yet at the same time, it's probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files, it contains a lot of the inherently complex work it's doing to this file, and it is so well and heavily commented that it should be pretty easy to ensure that any changes also keep the comments up to date (after all, any change without changing the resulting comments should show up like a sore thumb, and will most likely prompt the reviewer to look into it at the very least).

View on HN

> I love this! It's the "jazz music" of software development. Something which breaks all the "rules" but does so purposefully and explicitly so that it can become better than the "rules" allow.

I kind of see it as the opposite: “space shuttle style” is code that adheres to heavyweight rules that most software development has abandoned in favor of a more improvisational style.

But in either case it illustrates that code style rules are desirable, or not, based on the context; you need to understand what purpose rules serve and what trade-offs they involve to understand which rules to use; there's no one-size-fits-all solution.

I've had this latent thought for a while that I'm finally putting to words:

The complexity goes somewhere.

It's either into lots tests, or it's into something like shuttle style with lots of comments, or it's into a huge QA department, or it's into the type system / DB schema. It could even be going into the org structure!

But something, somewhere is handling the complexity and it is doing so as a partial function to the economic importance of the software to the stakeholders of the software.

I would say this is the fundamental problem of making software, and the reason that the whole world could learn JavaScript and yet being able to make great software would still be a rare skill and why even CRUD apps tax the minds of smart people.

Making the right component tackle the right amount of the right complexity is just a hard problem, and I think when it looks easy it's only because your options were constrained.

Probably the reason why libraries and frameworks are so omnipresent.

They solve the complexities the average programmer couldn't control on their own.

In the most ideal way, the complexity becomes the code structure. People can only keep track of 7 or so odd things in working memory at a time, we tackle complexity through chunking (abstraction), so that these new 7 things are in turn each 7 more things which are in turn 7 more things (ideally).

Your code structure is not my memory chunking structure and it will interfere.

At a point you run the risk of just overloading memory with way to many names and entities by merciless chunking, not to mention create a file or function maze.

Inlining works surprisingly well because you do not have to memorize things and can just read them. Theoretically it is a tooling problem, but nobody wrote a good enough "inline" tool so instead everyone relies on incomplete and buggy textual descriptions.

I'm not sure what to say, just do the right thing and be more practical than dogmatic I guess? There are situations where it's better not to install an abstraction. There are situations where an abstraction fits super well and is well understood by your dev team.

I had this thought when working with a legacy app the company was migrating to a modern platform. The legacy app was enormously complex and contained huge amounts of business logic. Migrating it didn't get rid of all that business logic. Now, using modern software patterns inside a framework with it's own complexities, it was a lot more organised but perhaps even more complex thanks to business logic now needing to fit inside the frameworks idioms and structures. It was nicer to read through, but no less complex.

In this case, the questionable choice of a "something" to handle the complexity is jarringly at odds with the high economic importance that the comments convey.

> tests

pv_controller.go has 1715 lines. To be generous, we might say half of it is comments. pv_controller_test.go has 359. Hopefully this code is exercised elsewhere in integration tests?

> a huge QA department

That's what you're signing up for when you choose a language that expresses cases mainly using if/then/else, and when you don't feel like testing every case.

> into the type system

Sum types aren't complex, although they're not familiar to everyone in the way i/t/e is. Even a result type (a straightforward example of a sum type, available e.g. in Rust) would simplify a lot of the "if err != nil" boilerplate, and greatly de-indent this code. It would probably also make invalid cases unrepresentable in a few parts of this file, eliminating a few more branches. In fact, a sum type typically requires an exhaustive pattern match, making practices such as their rule "every 'if' statement has a matching 'else'" the default.

My point is, complexity (and economic utility) are not conserved as you choose between these "somewhere"s. A few basic type system features can drastically reduce complexity for the QA team or eliminate comments and cases from the space shuttle.

Still - good on them for this degree of discipline & for so many well-written comments, with clear contracts about what is modified or not. ATBGE.

I don’t think this changes the inherent complexity of the code right? The basic logic of what is being done is still the same. De-indenting code but still having the same complexity, some of which is now abstracted by the language is still complexity. Maybe it somewhat helps the humans reading the code? But don’t you still have to reason about the basic state changes of the system the same way?

the code itself has the same complexity, but you are pushing the workload to the compiler, not to the human. Given the two options I typically prefer the compiler.

Compilers and libraries are written by humans and are probably the most fallible things ever, optimization wise. I'm not talking 1% here. I'd love to see an opposing compiler with an automated proof that it does what it claims.

Well, unless your computer can read and parse high order proofs over code. Not even Haskell can do that. (It can barely parse F types much less optimize them.) The closest I have come to that feature is Isabelle's simplify and even that is limited.

GCC and Clang can do basic bounds proofs at best and otherwise are tough buggy heuristic beasts. Just look at the trackers.

GHC has a slightly easier job but it's paid for in programming complexity. There are hoops in type system you have to jump to get performant code... and even then the code generated its at best meh quality. There is always some joker telling that it could be compiled for SIMD or multithreaded but that never materialized in usable form.

Compilers can only tell if programs are internally consistent, they can't help ensure that they are correct. Compilers don't know about inputs. Compilers don't know if the branch you wrote goes in the correct direction for a given input.

Well, the compiler can optimize away any superfluous branches which exist entirely to act as hints for the programmer.

> they can't help ensure that they are correct

Of course they can help. Maybe you meant:

> they can't ensure that they are correct

It prevents dumb mistakes like not covering all cases

Monadic Try, biased Either, Some, pattern matching and destructuting... you don’t even need to know category theory to use and understand them in code, and you let the compiler do all the tough work.

This example - despise the perplexing celebrations - is a product of the limits of Go

Yet you can argue that despite it's limitations they could make working (complex) software with it.

That's not an argument, that's a truism. You can make working (complex) software using nothing but machine code keyed into volatile memory via front-panel switches.

can you really?

Isn't that a case of "shit crayons"? People can and do make great things with terrible tools, but that doesn't make those tools any less terrible.

> This example - despise the perplexing celebrations - is a product of the limits of Go

All coding style is a product of the limitations of the language (well, except when it is coding style dictated by the limitations of a different language applied in a cargo cult fashion out of it's appropriate context); and celebrating the choice of how to approach this problem given that it was being done in this language isn't the same as celebrating the language choice. Go would be pretty close to my last choice for anything, but that doesn't stop me from recognizing this as a thoughtful way to apply Go to this problem.

So basically complexity would go in the language

A language that spends its complexity budget well can save complexity from a lot of programs. E.g. if you look at https://philipnilsson.github.io/Badness10k/escaping-hell-wit... , language A could offer all 5 of those ad-hoc solutions (making it a very complex language) and language B could just offer monads (making it a relatively simple language), but both languages would be just as effective in alleviating the complexity of programs written in that language.

And then language C(++) just skips unnecessary monads which are syntactic salt and uses exceptions, while providing the optional type for where you really, really want this behavior. Tradeoffs of course, but these were considered.

The main problem here with this "solution" is that any monad looks like every other monad. You can easily lose context and have to rely on naming conventions, which is quite terrible. You cannot spot what the code is doing at a glance.

And if you mess up, the type system and compiler will spout some completely unhelpful message because it parses everything the same.

Code that does very different things should look different, not identical. (Unlike what Lisp and FP people think.) Just think on why we do not use textual buttons everywhere in UI. It's a major reason fewer people accept Lisp than could...

> And then language C(++) just skips unnecessary monads which are syntactic salt and uses exceptions

Exceptions are an ad-hoc solution to a sixth problem. You still have the other five. (Ok, it's possible to use exceptions to replace null. But that still leaves the other four).

C does not have exceptions, or any solution to error-checking hell at all. C++ is a notoriously non-simple language. Neither is at all convincing as a counterargument.

> Code that does very different things should look different, not identical.

In a language with monads, code that does things that are specific to error-handling still looks very different from code that does things that are specific to async I/O. But code that does a thing that is fundamentally the same (parametric), such as composition, looks the same. This is the very essence of programming (and indeed of mathematics): "2 + 2" does something that is, on the surface, very different from "3 + 5", yet there is an important underlying commonality. Code that sorts a list of integers is doing something that is, on the surface, very different from sorting a list of strings, but there's an important underlying commonality. Monads just take the same thing one level higher.

> Code that does very different things should look different, not identical.

I think the major point you're missing here is that the whole point of pushing monads as a core abstraction is the way they shift what "different things" means.

This is a misreading, almost diametric, of what I wrote:

Me: "complexity is not conserved as we choose beween implementations"

You: "So basically complexity is conserved"

Maybe we're getting confused by this word "complexity", so let's break it down into two things: complexity of task handled (COTH) and complexity for programmer (CFP). For a given task of a given complexity, COTH is tautologically the same no matter how you choose to implement. The level of CFP, on the other hand, depends on how you decide to handle complexity. Implementing it in assembly language? High CFP. Implementing it with i/t/e, and commenting loudly your intention to handle all cases? Medium CFP. Enforcing these good, exhaustive standards with the type system? Low CFP. Why does CFP differ? Because the tradeoff matters. In particular, my point was that sum types add a tiny bit of complexity to the language, and remove a TON of complexity from the code. It's not a wash, or even a close contest.

(Notice that I don't make any super-radical suggestions, e.g. that their discipline around mutation/purity should be enforced with types or monads instead of exclamatory comments -- in which case, the CFP added to the language might outweigh the CFP alleviated from the typical file. But my point holds: CFP is not conserved)

> pv_controller.go has 1715 lines. To be generous, we might say half of it is comments. pv_controller_test.go has 359. Hopefully this code is exercised elsewhere in integration tests?

The PV subsystem interfaces with external storage systems, so integration/end-to-end tests are much more useful than unit tests (yes, they do exist).

The job is to distinguish between inherent complexity and accidental complexity.

I added this comment to my quotes collection.

I think you are more or less spot on. Personally I'll mention explicitly (one could argue that you already say this above) that well designed libraries and languages helps you by containing some of the inherent complexity while avoiding to add incidental complexity.

>Tesler's Law, also known as The Law of Conservation of Complexity, states that for any system there is a certain amount of complexity which cannot be reduced.

https://lawsofux.com/teslers-law.html

Wow, that site is beautiful beyond words.

I'd move the unnecessary big banner somewhere. A website is not a book and does not have cover pages.

Cute, but objectively not true.. using the right tool, or right approach can drastically simplify the solution, sometimes even making intractable problems solvable.

I don’t see how your example makes it false.

If it was objectively not true, then you could have infinite compression and any program could be reduced to a single bit.

If "conservation of complexity" were universally true then ANY compression would be impossible.

This isn't a dichotomy. My point is that there are clear examples of situations where you aren't just pushing complexity around, but actually achieving great simplifications.

>If "conservation of complexity" were universally true then ANY compression would be impossible.

No it wouldn't. The complexity of a pattern can usually be conserved while reducing its length, but for each pattern there is a limit. This is the entire concept behind the Kolmogorov complexity of a system and any patterns that cannot be reduced any further without removing complexity are at their limit already.

This is also related to the idea that you cannot have a universal compression algorithm.

I take this more to mean that the logic you're trying to implement has a fixed, non-zero level of complexity (sometimes called "essential" or "inherent" complexity), which forms the complexity floor of your application. On top of that, your implementation adds additional complexity (sometimes called "accidental" or "incidental" complexity), which is not-zero but not fixed.

So, my reading is that in saying "every application has an inherent amount of complexity that cannot be removed or hidden", the law is referring to the essential complexity. Meaning, the law says "some of the complexity is unavoidable in every application" vs. "the amount of complexity is fixed in every application". I do think the name of the law is a little weird, as it implies the latter meaning.

The trouble with Tesler's Law is that it's often difficult to distinguish between essential complexity and incidental complexity. When I face a UX problem that seems like an insurmountable barrier, if I step back and consider many perspectives, I often find a way to change the context so that the problem becomes easy to solve. What feels like essential complexity is often surprisingly incidental.

What I'm objecting to is the notion that there is a "fixed, non-zero level of complexity." Often real innovations in organization and structure allow for fundamentally simpler implementations. I'm reminded of physics: there's an inherent complexity in solving a rotational problem. But introduce polar coordinates and you fundamentally change the game, removing a dimension from your analysis for some problems, and making the solution trivial.

In this case your original estimate of the complexity was high due to a choice of coordinates increasing the incidental complexity of the solution

That's kind of a truism though. There will always be a non-zero entropy for the distinct kind of systems you want to build.

Unless there is one system to rule them all, configured exactly the way it is needed out the box.

Precisely, you can only move the complexity around but not eliminate it completely.

This^

> I kind of see it as the opposite: “space shuttle style” is code that adheres to heavyweight rules that most software development has abandoned in favor of a more improvisational style.

Most software other than that written for space shuttles and other seriously critical applications.

A colleague of mine works for a telco, and changes to software that runs on satellites takes months to approve and goes through verification stages that include running on simulators and duplicate hardware that is on the ground.

> probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files

I'm only halfway through John Ousterhout's book Philosophy of Software Design but I think it agrees with you on this -- that smallness-of-file or smallness-of-function is not a target to shoot for because it prevents the things you build from being deep. That you should strive to build modules which have deep functionality and small interfaces and should contain their complexity within them so the users don't have to know that complexity.

Now i'm squarely in frontend web-app development right now which definitely changes things (mainly the complexity is centered around enabling fast changes/additions to the codebase, and not the actual business logic for the most part), but while "deep functionality and small interfaces" sounds good on paper, most of the time giant files with a few functions exported aren't a good way to manage that.

Sure, it solves the problem when viewed from the outside. "users" of the software (users being other devs in this case) get a nice small interface and docs that explain how to use it, but internally it's much harder to work with. Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it.

In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. And it means that any devs that truly do understand it top-to-bottom would just waste a lot of time switching around files if it were split up.

Putting it all in the same file here nudges you to really make sure you understand it top to bottom before making any big changes. It's intimidating and scary for a reason, because at its core it is a complex piece of code, and dressing it up in "simple code's clothing" won't help.

> Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it.

There's a balance to be struck here; you want to minimize the size of the code a developer has to understand to work on (or with) a given abstraction, but you don't want to split beyond that point, as it only makes the developer jump around files. In my experience (with Java in particular), current development trends involve splitting the code too much.

Java (like many other languages of that generation) suffers from a lack of idiomatic 1:1 visibility. Whenever you split something up in Java it litters a namespace that is much bigger than necessary. Even private is too big when the class is full of tiny methods most of which most will never be meaningful to any of their peers except for that one call site. Sure, you can create inner function objects and with 8+ it's not even completely weird anymore, but that's still a far cry from the nested functions goodness of Pascal style "structured programming".

Agreed. I don't remember much from my brief exposure to Pascal years ago, but spending a good chunk of my programming years in Common Lisp has spoiled me; the simple ability to nest functions is something I sorely missed when working on Java codebases.

One of my favorite features of Haskell is that I can use where to write the auxiliary function definitions after the code that use them, but keep them scoped inside of a single function.

A relatively bad example here: https://wiki.haskell.org/Worker_wrapper#Hiding_the_worker

Bindings in the `where` clause can still access names in the main binding: https://wiki.haskell.org/Let_vs._Where#Advantages_of_where

This runs the risk of strongly coupling them to the main binding and making them less reusable. IMHO strong coupling where two bindings are aware of each others' internals should be avoided.

Like your link says: doesn’t lambda lifting solve that problem?

That’s what I always do if I want to move an auxillary function to the top-level for reusing elsewhere.

Indeed, I also prefer lifting, especially internally inside a module. It jives with the idea of loose coupling, strong cohesion.

> In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released.

I think you're on the money here. This was done intentionally, to ensure that devs understand the whole thing before making changes, because the functionality is so critical and so easy to mess up.

Also, breaking single logical things up into smaller files (rather than into a composition of smaller logical things) just leads to unneccessary file hopping and wrecks locality of reference for the dev.

Breaking things down into smaller pieces is good when it’s good... but there are drawbacks that I feel are often ignored.

For example, as you say, a small piece of stand alone code implies that someone can do meaningful work on it without understanding the entire context around it. It also implies that it is suitable for reuse. But if it’s both reused liberally and encourages you to keep working on it, it will almost certainly mean that your changes will have unanticipated consequences. So you still can’t get away with being unaware of the context.

When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later. Instead you keep working within the structure that’s already there, which often leads to concerns being spread out across different modules. Too often people decide on the structure before they even understand the problem.

I think it has similar problems as religiously following the DRY principle. There are so many situations where your code will be much, much worse if you insist on always sticking to DRY.

this.

> Too often people decide on the structure before they even understand the problem.

> When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later.

This is particularly true on teams where a significant proportion of the developers is reticent to refactor code as they go. It seems that given a developer with a sub-80th (or so)-percentile propensity to refactor, the more broken up the solution is (into smaller functions, methods, classes, modules, etc.), the less likely that developer will be to refactor the solution when an obviously better approach exists.

I just finished his book yesterday; he has a lot to say about size and comments. For size, your summary is spot-on. I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs.

As for comments, this file is essentially Ousterhout taken to the extreme. Still, I think he would have like it, given how critical this file is. In the book, he encourages writing more comments than the current trends would suggest, pointing out that you can't fully express abstractions in code, so all the things the code doesn't contain - the high-level concepts, the rationale, the caveats - should be documented in comments in appropriate places.

Overall, I'm extremely impressed by the book, and its focus on reducing and mitigating complexity.

There is a review and discussion of that book here, https://news.ycombinator.com/item?id=18331219

That is the exact thread that prompted me to buy the book.

> I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs.

Another way of describing this, that I ran across recently, is that this increases the cognitive load for developers working on the code, and cognitive load is one metric by which code can be measured as "good" or "bad". Things like excessive scrolling and switching between files increases cognitive load.

So heavily-commented code can decrease cognitive load if the programmer can read the comments and the related code block at the same time and if the comments help to explain why the code exists the way it does. Or, heavily-commented code can increase cognitive load if the comments don't accurately describe the code, or if they're so verbose that the programmer has to scroll up and down to digest both the comments and the code together.

> Another way of describing this, that I ran across recently, is that this increases the cognitive load for developers working on the code, and cognitive load is one metric by which code can be measured as "good" or "bad". Things like excessive scrolling and switching between files increases cognitive load.

Actually that’s one the reasons why he advocates not splitting code (arbitrarily) in the book.

Scrolling is fixable by code folding. You've read and understood the part, now you can fold it. It takes some factoring discipline to not end up with a woven structure.

Unlike a single use function, it does not have a name to remember and is in-place - and definitely is not shared so can be assumed to be safe to modify.

Comments would be much nicer if we could still use column-based commenting which unfortunately is not usable in any modern IDE. Reading code and comments side by side tends to work much better than interleaving.

I really enjoyed the Ousterhout book as well. It's my current user manual for reviewing pull requests.

Putting my PR reviewer hat on I would say the code in question would pass muster if it were relatively stable so you would not be constantly redoing the comments. I love nice comments but Golang does not give you much help keeping them in sync.

(Weirdly enough I'm working on a PR for persistent volume documentation at VMware today so the code is very apropos.)

> Golang does not give you much help keeping them in sync.

What do you have in mind, here? Something like python's doctest?

Elixir tests code in comments for correctness

Javadoc. It generates warnings if method argument documentation does not line up with the method signature, for example.

Martin Fowler of the Agile world, and Garret Smith of the Erlang community, are both excellent programmers whom I respect, and they both take the approach of breaking code into lots of extremely small functions.

Having tried that style, I notice that I don't particularly favor it, and for the very reason you site: the code is no longer all in one place.

I've switched to moderately sized methods/functions with comments every few lines. Some say that comments like this are a smell, and that you should refactor the commented section of code into it's own function, but honestly comments are easier to read than method names (and again, there's the benefit of locality).

I'll have to take a look at the book.

A technique I use quite a bit is to group functionality within a method using `#{` and `#}` to bound the code doing the thing. It gets you the "grouping" idea of lots of methods, but if the code is only used in one place, there is no reason to pull it out into a method.

Something like:

  #{ Parse input parameters
  .... <several lines of code here>
  #}

I like this idea and will try to apply it to our code. Why do you prefer it over splitting the code out in a different method? Is it so you can read everything in one glance? If so, an ‘inline’ feature could be added to IDEs to show definitions inline.

Also, when using applicative or Monadic style programming there is little reason not to split things of in small separate functions and chain them together in the right order.

Yea, it probably doesn't make sense for every coding style. I just find that if I split things apart too much it turns into spaghetti that I have to try to weave my way through to understand what's going on.

There are definitely places where it works out really well though.

I assume everyone who splits code into smaller pieces use modern IDEs that makes it trivial to navigate to functions by clicking them etc.

I say this because I'm always astonished by the number of "modern" programmers who refuse to use IDEs.

IDE or not, jumping around between functions amd their callers to understand a process is annoying.

Sure.

In sane code the name of the function should describe what they do well enough that you rarely have to click in to learn how they do it.

Or something like that...

That applies recursively to the function you're just reading :).

I.e. I wouldn't be inside a particular function of a particular module if I didn't have to know something about its implementation. There's a good chance I need to understand all of it at the level of abstraction of the module (often because I'm supposed to change something about it). Making that less painful leads to better and less bug-inducing experience.

Elsewhere[0], 'usrusr brings attention to nested functions, lack of which I see as a huge factor contributing to overeager splitting of code. With nested functions, you can have "best of both worlds" - a function whose implementation is divvied up into well-named pieces, while keeping those same pieces in the correct conceptual place, close to where they're used, and restricted from polluting unrelated code.

[0] - https://news.ycombinator.com/item?id=18773691

Totally agree on nested functions. I currently have to deal with Java, and those are the biggest thing I miss from Python.

I would make no claims to being an exceptional programmer, but fwiw I don't like nested functions - it always takes me a lot longer to reason about what a function is doing, when it has functions defined inside it.

This can depend on the syntax and semantics of nested functions.

Pascal nested procedures are pretty easy to parse and if they're defined before the `var` block then you don't need to worry about non-local state modification (apart from the parameters, but modifying parameters is unusual).

First-class nested functions with variable capture are harder to understand. Nested functions in e.g. JS are more like object construction than function definition, and it's generally expected that such nested functions will be capturing state from the enclosing context.

Standard Pascal permitted outer scope variable access combined with downward funargs - the nested functions could be passed to other functions, but could not be stored in variables or returned as arguments. This reduces the non-local effects, and handily doesn't require a GC to ensure memory safety, since the captured state can stay on the stack, because the nested function won't outlive the frame.

For nested decomposition of a problem, I'm more of a fan of Pascal-style nested procedures than nested function expressions like in JS. Idiomatically, one expects function expressions to capture state, while nested procedures are just more procedural decomposition.

I second the answer about having them before var-block: In python I always follow the style of having nested functions directly after the outer function header, and only going one layer deep. This way the only thing in their scope is their parameters as well as the parameters of the outer function. Using this restriction I find them very useful and concise - much better than having one-off helper functions in the outer module scope.

Kotlin supports nested functions along with quite smooth Java interop.

You seem to be arguing against the concept of functions in general here.

That's probably not what you mean to say, but I don't find much to really discuss here.

I do like local functions in Python, but I don't see a huge difference from having the called function right below the calling one.

I have read an extreme counter-opinion in some J or APL article. It said that it is a bad pactice to name small and common functions:

The example was maybe the average function - and the reasoning was, if I recall correctly:

1. The defintion is shorter then the name average ;)

2. Every praticioning programmer will recognise the definition as a common idiom

3. From the definition it is immediately clear how corner cases are handled (e.g. zero length array)

I just leave this here as an example to show, that programming communities/cultures exists with completely different/alien? ideas about what clean code is ;)

Allegedly. And then you suddenly want to write a SIMD version of an average or optimize it... Mass search and replace time? That'd bloat the code a lot.

Common repeated well defined and mostly immutable code is best left as functions. This is why for example in C strcmp exists instead of everyone writing a two-liner - and the specialized variant gives big performance gains.

I've got your point.

Just for the sake of nitpicking: both your actual examples have been solved by compilers automatically.

SIMD/AVX: https://code.jsoftware.com/wiki/Guides/AVX

JVM generates vectorized string instructions from your simple for cycles - if availabe: http://jcdav.is/2016/09/01/How-the-JVM-compares-your-strings...

edit: my second example link is maybe wrong, but anyway, auto-vectorization is a real thing...

> I assume everyone who splits code into smaller pieces use modern IDEs that makes it trivial to navigate to functions by clicking them etc.

That's... not the point. Jumping around is. Imagine reading this comment thread on a bizarro-HN, where you only get to see a short camelCased summary like: debunk(this.previousComment), and have to click to open each comment in a new tab. This is how jumping around small functions feel.

> I say this because I'm always astonished by the number of "modern" programmers who refuse to use IDEs.

There are reasons for it. Many languages don't have an IDE. Many are not suitable for one (especially ones closer to Lisp on expressiveness spectrum). IDEs are heavy and often mouse-oriented, and not efficient for reading and editing text. Sometimes (read: Java) they are a crutch to work around the expressive deficiencies of the language.

Mind you, I have nothing but good things to say about IntelliJ. I've spent a lot of time in it even recently, and I pick it up any time I have to do anything in Java. But for everything else, I launch Emacs, because it can handle all other languages well, and has superior editing capabilities.

> That's... not the point. Jumping around is.

There is a tradeoff: Small functions make high-level logic clearly visible and easy to find, at the price of forcing you to jump around when you want to dive into implementation details. Putting everything into one big function lets you follow all the implementation details without jumping, at the price of making you read everything to actually understand what the code is doing.

The latter is the biggest price you can possibly make me pay. Jumping around is a minor inconvenience.

Wasn’t that the purpose of the long comments: To sketch out the high level logic, while keeping all the code in one place. Jumping around, you now have to do that to be sure of the details and the complexity can be hidden in layers of functions. Complexity that often matters.

By choosing the right names for a function or class a lot of jumping around can be prevented. Most IDEs also have a feature (including key combination) to show the documentation of the method/function. The only reason that remains is when you doubt the correctness of the method and need to look at its definition.

The only and critical reason - skipping the check will net you tons of debugging all the time because hidden assumptions are often not documented. (or otherwise visible)

You're describing an unstructured single function.

There's a middle ground - sectioned code, esp. with code folding or similar commented sections. Good IDEs support such a feature so you can hide the code you think you understand to not look at it every time.

You will have to read this code at least once or trust the documentation and name. I found that in "mature" code the latter is a recipe for disaster and days of debugging.

> Many languages don't have an IDE. Many are not suitable for one (especially ones closer to Lisp on expressiveness spectrum).

This is a minor point of your comment, but I'd like to refute it: Lispers actually often cite IDE integration as one of the great features of Lisp. It may have lost pace a bit with some of the very best modern ones, but Lisps have had "modern" IDEs since about the 70s or 80s, with stuff like jumping to function definitions, finding usages of a function, looking up the docs etc. Even better, this is usually a part of the language runtime (predating most other language servers by quite a bit), so it can even apply to dynamic uses.

For Java programming, I definitely use IDE. Even for Erlang programming I use Emacs with EDTS to get a more or less IDE experience.

I'm saying that, even with that convenience, I don't wrapping a line or two in it's own function is more readable than just adding a comment above those two lines within the context of a (somewhat) larger function.

In general I prefer smaller functions. I think the threshold is when they become hard to name. Then I will step back and think about whether it's worthwhile.

I think it might come with experience. I definately gravitate to very small classses/functions, personally prefering classes to fit within a single screen.

Obviously this isn't always achievable.

It comes down to: "What do you want to focus on?" Each drill down should be to a lower level of abstraction. It is seperating the what from the how.

It results in functions/classes that either detail a flow (set of decisions) or that implement actions. For example I generally don't need to know how to read a file from disk where I am trying to decide if I should read a file.

When looking at code you drill down in a very vertical fashion when you want to know "How does it do operation X?" Meaning the operation largely can be understood in isolation. There shouldn't be a lot of context that is required for it. A side-effect is that all the unit tests are very self contained.

This utlimately steers you towards a much more functional and compositional programming style. Certainly it is moving complexity around, but the end result is heavy compartmentalization and limiting the context required to understand any section of code.

I like your description of separating flow from actions, I try to do that too. I want my functions higher up the abstraction stack to read like they are orchestrating black boxes of functionality.

> that smallness-of-file or smallness-of-function is not a target to shoot for

I disagree. Unless you are methodical and know what you are doing (like NASA or the authors of Kubernetes), it is hard to create large functions and files by keeping levels of detail consistent and not repeating code.

How do you test a function that has 100 unique outcomes? How do you safely maintain it to ensure it won't break? How do you even know it's working?

Write 100 tests?

Write automated proofs? Probably best but tooling is junk - it's rarely supported.

Constraint or property based tests as opposed to value tests? (That depends on how unique.) Usually the only thing that does unique things is a direct mapping, everything else is either an equation or has other preconditions, postconditions and internal or external properties. It also makes the logic more directly visible and refactoring easier presuming tests are reasonably written.

Ensure coverage with a tool to be sure.

Haven't read the book. Goes to my Wish List. I completely agree about encapsulation of complexity inside the module.

Similar to UX design principles. Just like google or other successful sites present a very simple interface (in case of google the home page is quite clean) but behind it lies very complex code.

IMO, encapsulating complexity and handling all edge cases also lends to total functional programming [1] even without using pure functional languages. [1] https://softwareengineering.stackexchange.com/a/334876/38285

> That you should strive to build modules which have deep functionality and small interfaces...

this. interfaces are not the most important thing in a software but definitely one of the most. you can live with a crappy implementation but your interfaces must cater to the usecase and should only hcange when core premises change.

I agree. My day job is working on code that isn't this level of critical, but also has the characteristic of being low level, both closer to the metal than typical backend code and also called by so much frontend and backend code that if there was such a thing as "even backend-ier code" this would be a good example. If you miss a nuance, a horde of angry developers will show up at your desk the moment the build deploys, and if you don't fix it fast, that turns into a horde of angry customers.

With code that far down in the inner loops of mission critical code, the dominant time cost isn't really writing or even understanding the code, it's ramping up on the nuances of the scenarios you serve, checking the direct effects of your changes, and then checking the second order and third order effects of your changes.

When your code is so far down that every change you make has third order effects, it's a lot easier to maintain code that is meticulously commented and written for maximum thoroughness explicitly baked into the file in front of you, because you're much more likely to miss the nuances if you just finished piecing together ten different files to figure out how the happy path works.

Note to others: 3rd order is not necessarily 10^10^10. It can easily be 10^100^1000.

Interesting! Is there a convention to follow?

I hope you mean x (multiplication) rather than ^ (exponentiation). If we're talking about fanout (each of 10 items of has 10 subitems, each of which has 10 subitems), multiplication is the relevant operation. And 10^(10^10) or (10^10)^10 is a hopelessly, uselessly, inconceivably large number. (Exponentiation is not associative; 3^(3^3)=3^9= 19683 while (3^3)^3=9^3=729.)

I took it to mean something fairly vague about potential super-exponential blowup in the complexity of the states you have to reason about, when you have to reason directly about third-order effects in a code base.

Well said. I feel that 80% of this thread is people talking past each other, with different assumptions about what the code in question is going to do.

I completely agree. For code that is unavoidably complex, I love this style too.

I am all for code that is concise and whose syntax/naming is expressive, but sometimes comments are necessary to clearly spell out the logic or business use case. Expressive code can only go so far. Well-crafted comments significantly reduce the amount of time required for other developers to dive in and become productive with an unfamiliar code base.

The key is keeping the comments up to date. There's nothing worse than an inaccurate comment. One's code review process must include a review of the comments accompanying the modified lines.

Agreed. Also, comments that emphasize the "why" over the "what" are not obviated by descriptive names.

I've found it very helpful to rubber duck comments to get into the right why mode. Consider how you'd explain this piece of code to a less experienced guy on the team - and write down exactly that. Why was this added? Why was the old thing in place changed, what broke and had to be changed? Why didn't you do the other obvious thing? And remember - usually you should explain something until it's clear for you, and then one more step.

Outdated comments that explain the business use case or purpose are still better than no comments. It gives you background information how the code evolved or what it was supposed to do.

It's probably because reading comments only is worse than reading code without comments, that some devs developed an aversion towards outdated comments and thus comments in general.

Comments are additional information and no source of truth, always take them as that and read code and comments.

It’s much worse in codebases that predate version control. At least a commit shows the context of why it was added.

Yeah and over the course of 2 or 3 decades of development a lot of software has moved through several different version control,ticketing systems and developers.

So you wind up with files stating an author who no longer works there with an email address that the company used 3 acquisitions ago, a ticket number you aren't even sure what system it was for but you just know it isn't being used anymore and source control history that goes back 5 years out of a total 25 years of development.

The entropy is real.

You just described LibreOffice!

I just described a lot of offices.

About five years ago I worked on a codebase with a similar bit of code. It wasn't nearly this big, but it was branchy, procedural, and verbosely commented. I didn't write the initial version but worked on it quite a bit and learned to appreciate the advantages of the style for the nasty bit of logic it implemented. I ended up having to vigorously defend it against another developer's half-cocked attempt at "refactoring", which got deployed more than once (behind my back) and thankfully broke in pretty obvious ways each time.

I warned him at the beginning that he needed to spend a couple of hours wrapping his head around the code before trying to modify it.

I warned him that the code arrived at its current state after long and painful experience.

I warned him that other developers had worked on the code, had exactly his initial reaction, and eventually admitted they couldn't find a way to improve it.

I warned him, after he confidently declared that it just needed to be "more object-oriented," that I had written a lot of object-oriented code, was open to writing it in an object-oriented style if it would improve things, and could not think of any way to do so that would not make things worse.

But he could not even bring himself to read the existing code. He never did. He worked on it for three weeks, wrote thousands of lines of code, and even tried to deploy his own version without ever spending a single contiguous, focused, hour-long block of time reading the existing code. He was "refactoring" the whole time.

The code he produced was exactly how you imagine it. The code went from several hundred lines in a single file with no classes (just a containing class acting basically as a namespace) to thousands of lines scattered over half a dozen files with at least that many classes. The guy kept adding classes and kept adding test after test after test. He couldn't get his code to pass the test cases I had written, so several times he declared that my test cases were wrong and changed or removed assertions. He also claimed that the existing code couldn't "pass" his tests because he had hundreds of lines of tests for components that only existed in his code. According to him, this proved that there was a ton of "hidden untested functionality" in the existing code, which was therefore "dangerous."

I was pretty busy with other things, and this was his project now, so if he had been a little bit more clever he probably could have made his abomination stable enough to replace the existing version over my objections. Thankfully it was never good enough to survive in production.

Honestly, after the experience described, I would consider the person a work hazard and try to make sure they didn't touch anything business critical.

I think he realized his comfort zone is the middle tier of three-tier web apps and is doing his best to stay there.

On the other hand, this code was in the middle tier of a three-tier Grails web app, so....

Why do all anti oop posts sound like completely unlikely, exaggerated lies? A few hundred lines to thousands with dozens of files and classes (gasp). Lol

I don't have anything against OOP as a style to have in your repertoire and use when it's beneficial. Unfortunately, OOP was promoted in a pernicious way for a couple of decades, resulting in stories like this where a guy who

* didn't understand a difficult piece of code

* knew that the code was written and maintained in its current form by multiple programmers known and respected by him who had more exposure to the problem than he did

* knew that said programmers were well-versed in OO programming and had considered and rejected OO style as a way of improving the code

... this guy, because he had learned software development in OOP's heyday, simply knew as a self-evident truth that the code ought to be OO, because OO was better, and if the code was hard to understand then the proper way to approach it and try to understand it was to incrementally refactor it into classes. And he never stopped believing these things even after his multiple failures.

This is not a knock against OOP. This is a knock against the people in the 1990s and 2000s who thought software engineering was a solved problem, thought class-oriented OOP (exemplified by design-patterns-driven corporate Java style) was the solution, and thought the only hard problem left in the field was to teach all programmers to understand and accept these truths.

> A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements. A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements, has a lot of "pointless comments" that just describe what the line or few lines around it is doing, and has a lot of "logic" in the comments which could quickly become outdated or wrong compared to the actual code.

I actually understood it immediately when I read it. There's something to be said when you have a decent roadmap for debugging critical code segments. It's not just software maintenance here, it's for the "holy crap something went wrong and we can't understand how we got here!" moments.

This is why I prefer older JavaScript to the more recent stuff, with transpiration pipelines and AI-like optimizing compilers that gnaw on giant state trees.

That stuff is lovely and terse when it works. And when it doesn’t you are at a total loss.

The beauty of imperative code is there’s a beginning and an end, and at every step you can easily see what follows and without too much work what came before.

I don't understand the objection to having more, smaller files, at least in Go where they can all be in the same package.

Once two functions are too far apart to be on screen at the same time, jumping back and forth between two functions in the same file doesn't seem any easier than switching between different files. If anything, switching between two different files is easier since they each get an editor tab.

On the other hand, being wary about extracting functions (which moves things logically together further apart) makes more sense.

For me, it's less about number of files than it is "hoeany files do I have to open to figure out how something works? How many levels of indirection do I have to keep in my head?"

I started out writing low-ish level code. Motion control, image processing, digital imaging, and the application level code that coordinated it all. I've steadily moved up the abstraction tree over the last 13 years and there's one thing I hate about it; too many fundtions and classes which do far too little. Abstraction for abstraction's sake.

It's a bit subjective of course and not everyone has their own style, but there is a Starbucks difference in philosophy between the two types of groups.

> [how many] files do I have to open to figure out how something works?

With jump-to-definition editor integration, who cares?

Anyone who actually worked with code requiring so much jumping.

I did, in an IDE, and I can tell you, jump-to-definition removes the problem of "what file do I have to open now?", but still leaves you with the questions like "where am I?", "how did I get here?" and "what was I trying to understand, again?", which you start asking yourself after ~sixth jump.

> "how did I get here?"

In emacs go-mode, pop-tag-mark will take you back to where you jumped from.

I know.

It'll take me one level up, which is not sufficient to answer my question, and by the time I find my bearings again, I have to jump back down a couple levels.

I've done a lot of this, both for Common Lisp codebases in Emacs, and for Java codebases in IntelliJ. Having to jump around and remember stuff eats into "7 ± 2" short-term memory limit that you need for the code you're working on.

(With Emacs, it's at least a tad easier to split the window to have 4+ different in view at the same time.)

Yeah, I like to use a couple of big monitors plus my laptop screen, and have all the relevant code visually accessible at once.

Not to language bash, but situations like this are why I avoid "enterprise" patterns and abstractions. Not that they aren't ever useful, but they're often implemented before they are needed, or not needed at all in practical terms. Some C# and Java tend to be some of the worst examples I've encountered. Even if the languages themselves can have much simpler implementations.

Java has these scaffolding developers who spend great deal of time in writing, 'ticket booking', 'toy/video stores' etc with all technology, framework, libs they are looking to learn. None of this code does anything useful but developers add to their resume claiming they developed reservation systems and e-commerce platform with mongoDB or whatever.

I'd add this kind of bullshit started directly from Sun Microsystems when they gave us Java EE blueprints to develop end-to-end enterprise solutions in all seriousness.

"far too little" it sounds good but you need to think about what you're saying. Do you want a function to be doing too much or too little. I think any sensible programmer will choose too little. It's easier to understand. This isn't about abstraction and the trendy fashion that is the irrational fear of it nowadays. Is about people being better at tackling problems in small doses.

You should learn markers in vim; really useful for jumping between marks in the code

This is what a lot of Go code looks like. This "space shuttle" code honestly isn't much more verbose than most Go code I interact with. The main difference is they have more comments here.

"else/else if" is not used very often in Go.

It's not used very often here, either. Despite their claims at the top of the code of every `if` having a corresponding `else`, there are 140 `if`s, but only 20 `else`s and a mere 5 `else if`s.

Most of the ifs are for early termination. Using an else on one of these is wasteful.

Likewise, I've seen a lot of Go that is far away from the idomatic Go heaven that is the standard library. Although that happens in every language as soon as there's reasonable complexity.

As for the "space shuttle" term, it's more of a euphemism for "do not even attempt to refactor this mess, it's so complex that you'll surely fuck up if you do, and you'll make it harder for the one guy who can actually understand this"

It's freaking kubernetes, so obviously it will be a well understood file by a LOT of people. However this is the kind of file that also exists in unknown, non-open source products, and while may have been understood by lots of people (at the respective companies) sometimes a period of 5 years of inactivity and layoffs and hiring goes by... By gosh darn it- this is it. It's the reason we have shitty code but great products.

Straightforward concepts should be coded consisely and complex concepts should be coded verbosely. The commenting in this file slows down the thinking of the reader to an appropriate level.

To be honest, I remember myself back during my first dev seps struggling a lot reading code since a lot of the senior programmers tend to code very effectively, so it was extremely hard to follow an end-to-end solution without terribles headaches.

This is just brilliant, not only because as educational exercises explains perfectly what's doing but the business/thinking process/context.

Never read 1k lines so quickly before, never enjoyed so much.

> It's the "jazz music" of software development.

If Anthony Braxton wrote software...

oh no the "I love it" first post ... the fact this type of post is the top of so many threads leads me to believe theres more behind these posts.

the positive renforcement cheerleading to start off so that people dont get demoralized in the hateful comments, while effective also seems fake.

I also noticed this. I wouldn't be surprised if there was some editor-in-chief who basically scores the comments based on his taste.

Reply on news.ycombinator.com