Hacker News Reader

Comment by agentultra | Hacker News Reader

Ah, the ample club of wishful thinking.

There are two general ways of approaching software design (and I'm paraphrasing Tony Hoare here):

1. You can write software so simple there are obviously no errors

2. You can write software so complex there are no obvious errors

One thing that escapes "grug" is that achieving 1. often requires more sophistication than their magical club allows. Most well-intentioned "grug" developers will write software so simple that it becomes it's own form of complexity: a giant mud-ball of for-loops, while-loops, variable assignments, and other wonderful side effects. Instead of addressing complexity head-on with abstraction, "grug" will beat "galaxy brain" over the head.

What grug fails to understand is that simplicity isn't easy or familiar. It doesn't mean "sticking to what you know." If often requires being able to reason about programs and to verify that reasoning!

But go ahead grug... keep beating people over the head with the club and hope that the complexity will go away.

View on HN

As usual the truth is somewhere in the middle. I’ve seen nearly $1m in development cost and a year of time lost to the complexity demon due to big brains playing AWS Service Bingo while a competitor hired a single grug to do the same task.

That single grug got it done in ~1 month for basically nothing, and without the multiple AWS service overhead it ran much faster, fewer resources, and dead simple to maintain. Bigger company bought the smaller one, then proceeded to toss the grug code and continue the big brained approach, as far as I know never reaching parity.

But there were cool network diagrams, data diagrams, and all sorts of new, interesting, and complex technology developers enjoy playing with.

I’m more inclined to side with grug now.

The OP's point is that being able to produce a simple solution to a complex problem only seems like the product of "refusing to be smart" from the outside. From the inside, there's always a delicate balancing act - each step involves choosing approaches that may generate complexity just in different ways. The same data used in different places in the application? What do you do? Objects? Globals? Cut-and-paste. Any of those solutions might be fully justified or might degenerate into a tangle by the time the application is done. Once, if you have a nice, simple working application, you can say "it's all just 'grug'" but that's essentially what they a call "humble brag".

Undoing someone's copy-pasted for-loops and while-loops is called "factoring" and undoing someone's bad abstractions is called "re-factoring" and is about 10x as slow. At this point in my career I have seen way more crappy abstractions and needlessly abstracted code than grug-brain code.

And, to be brutally honest, as much as I love those functional combinators, first-class functions, streams, etc, they suck to reason about.

Sometimes loops are better!

I think you found the head of the nail, so let me hit it:

”Simple” code is simple to reason about, but it is not as expressive.

”Complex” code is just as simple to reason about, but being more expressive, it requires more intimate knowledge of the language features and abstractions used.

Then there is ”bad” code, which is confusing for reasons other than domain complexity.

Sometimes complexity just gets handed to a developer even if they say no, and that doesn’t make their complex code bad code.

> ”Complex” code is just as simple to reason about, but being more expressive, it requires more intimate knowledge of the language features and abstractions used.

> Then there is ”bad” code, which is confusing for reasons other than domain complexity.

I've previously seen these two summarized as "the difference between complex and complicated".

I personally almost always use loops (in Java) unless I know the dataset being dealt with is small, unless I'm writing stuff in Scala where maps and flatMaps are first class, but diving into Scala is its own can of worms.

> And, to be brutally honest, as much as I love those functional combinators, first-class functions, streams, etc, they suck to reason about.

> Sometimes loops are better!

That I think is backwards. A loop could be doing literally anything - it probably is futzing with global variables - so there's no way to reason about it except by executing the whole thing in your head. A map (or mapA) or a fold (or foldM) or a filter or a scan is much more amenable to reasoning, since it's so much more specific about what it's doing even without looking at the body.

I like to write loops where they follow the functional map/filter/reduce paradigm where they don’t mutate anything except some initial variables you “fold” over (defined immediately prior to the loop) and which are treated immutable (or uniquely owned) after the loop.

I find this has good readability and by containing the mutation you can reason about it “at a distance” quite simply since further away it’s for-intents-and-purposes pure code.

What you might lose are the compiler-enforced guarantees that a functional language gives you. Some languages give you the best of both worlds - with rust you could put this in a pure function (immutable inputs, owned outputs) and the borrow checker even reasons about things like this within a function body.

I think a loop is ok if it only modifies variables whose scope is the loop.

And that's what map() basically does.

Not true in reality, so your theory must be wrong.

I've been doing this professionally for over a decade and it's been true in my experience, shrug.

> it probably is futzing with global variables

The opposite is true in my experience. Loops working over variables local to a function is a lot easier to reason about then a series of intricately intertwined lambdas, if they share state. If everything you do is functional then a series of pipelines might be alright, but it might not. If the functions are spread all over the place it can take a larger amount of mental load to keep all the context together.

Code should permit local reasoning, and anytime that is obscured, rather than helped by, abstractions, we incur additional cognitive load.

> Loops working over variables local to a function is a lot easier to reason about then a series of intricately intertwined lambdas, if they share state.

I mean accessing a mutable variable from a lambda is obviously insane, agreed. The whole point of using the functional combinators is that you don't do that.

I think we probably agree in a lot of concrete cases but discussing in the abstract tends to bring out polarizing statements. That's why I qualified most of my remarks, because I'll happily ham it up with a functional style...in tests, and other places where I don't feel strongly about what's going on.

When you are forced to use some accumulating global state, that leaves you with writing in a different style--loopy, if you will--which maybe is a good signal to the reader that something is weird or different, but then again maybe it isn't.

One thing that bit me using Java streams recently is that it completely broke down when I had a concurrent data structure that needed to have the coarseness of locking tuned for performance. Laziness and functional streaming operators had to just go out the window to even make it clear where a lock was taken and released. So loops it was.

> One thing that bit me using Java streams recently is that it completely broke down when I had a concurrent data structure that needed to have the coarseness of locking tuned for performance. Laziness and functional streaming operators had to just go out the window to even make it clear where a lock was taken and released.

Well yeah, that's very much expected. If you're even talking about locking, stream transformations aren't a good fit (except maybe if you do the microbatching/orchestration style where your stream transformation steps operate on chunks of a few thousand items - and even then your locks should be scoped to a single transformation step).

(Now I'd happily claim that for equal developer time one can usually outperform a locking-and-mutation implementation with a stream-processing implementation - not because the low-level mechanics are faster but because it's easier to understand what's going on and make algorithmic improvements - but that's a rather different matter)

I've seen a massive, "simple", abstraction-free function of 500-1000 lines take smart engineers quarters to safely "factor". I've seen this same thing happen a handful of times. Big spaghetti function requires someone to build a new test suite from scratch so that they can refactor safely, because everything ends up having spooky surprising distant effects, because there isn't any abstraction to contain things.

They're dead simple, there's no abstraction, its just loops and variables (and state, which is the real killer). But they're impossible to reason about as a result.

Sure, this happens. But imagine it was 10 times that size and factory poorly. If you had the same lack of tests, I'd contend it'd be a worse nightmare, because you have to first reverse engineer the confusing old design, unconfuse it, and then incrementally refactor it.

You can write bad code in any language with any constructs. The more code you do write the worse bad ideas just metastasize like cancer.

> Instead of addressing complexity head-on with abstraction, "grug" will beat "galaxy brain" over the head.

the argument made in the piece is more nuanced than that. The author points out that you often cannot address complexity head on (in particular not with abstraction), because you don't even know what your complexity looks like, as the author says, complexity isn't trivial to see.

This was the old problem of inheritance as a paradigm which tried to anticipate structure of programs and code when often you can't anticipate what shape your program is going to take, often leaving you in dead ends and wrong hierarchies or taxonomies.

The author isn't saying to not abstract at all but to not do it early. Casey from Handmade Hero had a similar style he called, IIRC 'compression' based programming, implying that you write your code, and as you go over it again and again you see where duplication pops up and only then you factor it out into some higher abstraction.

This. Develop the logic and flow for all of the thing, make it big, code it simple and even redundant, but really basic and simple. That's your "proof of concept". Pass two is finding and factoring out all of the redundant stuff into functions and libraries, grouping like with like. It's like in relational database design - first you grab all the fields needed, then group them and normalize them.

I can see where one can build upon that intuition. If one replaces “abstraction” with “indirection” it becomes apparent what is happening.

When programs are defined in terms of operational semantics, that is procedurally, it must still be reasoned about in some way if your goal is to write a program so simple there are obviously no errors.

Patterns of indirection are not abstractions. It’s a fine practice for controlling indirection in code! And highly effective. But it’s not what I mean by abstraction.

Abstraction enables new layers of semantics by completely encapsulating complexity. Think of function application in almost any procedural language that is compiled to some target machine language. The compiler has to generate a lot of code in the host language to manage memory segments, return pointers, etc. As the programmer using the language you don’t even think about it at all.

One doesn’t arrive at such an abstraction by writing a bunch of spaghetti code and encapsulating common patterns.

And that is often how one arrives at simple code. It’s not “simple” because it doesn’t require anyone to learn anything or work hard to understand it. It’s simple because once the abstraction is established and proven you’re free to think about bigger, more interesting ideas.

> One doesn’t arrive at such an abstraction by writing a bunch of spaghetti code and encapsulating common patterns.

No you don't get an abstraction like the one you described. But you often can hide a huge mess of complex code behind an interface and make it simple for your part of the system to deal with it. e.g. Think of the interface to a search engine or a machine learning system.

> It’s not “simple” because it doesn’t require anyone to learn anything or work hard to understand it. It’s simple because once the abstraction is established and proven you’re free to think about bigger, more interesting ideas.

In many systems there aren't any more interesting ideas and in that case making an abstraction where you work hard to understand it is a liability. People don't have time for that. People particularly don't have time for changing it when the outside world changes such that the abstraction no longer fits. And the outside world changes quite frequently.

Whoa now, I was with you until you seemed to imply that "simple" software means shitty code. Simple in this context means a solution with as little complexity as possible. Hopefully written elegantly and carefully, with lots of thought for future maintenance and readability.

Maybe you use a 10-year-old, constantly maintained and thoroughly tested library to do 90% of the work instead of writing everything yourself.

Would it be faster to compute some end state without intermediate calculations? Probably. But how about we just spin a loop forward instead if it's more accurate that way and easier to understand.

What if we cached values, then made sure the caches are always in sync? That should speed things up. Well, maybe we'll write it without caching and see how that goes first.

How about an intricate system of locks and serializable transactions so that multiple requests can run at the same time? Or maybe we just queue things up and run them one at a time.

Nothing to do with nested for loops.

> Hopefully written elegantly and carefully, with lots of thought for future maintenance and readability.

Those are fine things to value even if they are highly subjective.

You might think it’s easier to read a while loop with a handful of mutating variables and a few branches. I might disagree.

I think it’s easier to read an accumulating function that uses traverse.

It’s wishful thinking that we should both hold the same values or that only one of us is right.

> big brain type system shaman often say type correctness main point type system, but grug note big brain type system shaman not often ship code. grug suppose code never shipped is correct, in some sense, but not really what grug mean when say correct!

This big brain ships code every day.

Also streams big brain work in big brain language once a week to show people that it’s not the work of big brain shamans.

Big brain is happy to answer questions and help people learn.

> a giant mud-ball of for-loops, while-loops, variable assignments, and other wonderful side effects. Instead of addressing complexity head-on with abstraction

I very much prefer to work in a codebase of poorly written loops and variable assignments rather than one with poor abstractions.

Poor abstractions can harm way, way more than spaghetti code. Their harm usually spreads out through system.

Imagine this:

* 1 single poorly written function which takes a number of inputs and has an output, but internally it's long and messy.

* A bunch of abstracted entities, which interact throughout the system, but are poorly designed.

The complexity of 1 is isolated to the implementation. There's a nice encapsulation. Removing/Rewriting is easy.

But 2 becomes a complete mess which is not isolated. You'd have to skin it layer by layer. Thats way more scary to me.

>There are two general ways of approaching software design (and I'm paraphrasing Tony Hoare here): 1. You can write software so simple there are obviously no errors 2. You can write software so complex there are no obvious errors

And both are neither here, nor there, regarding what the article talks about, which is basic advice that always holds: avoid complexity, say no whenever possible, opt for 80/20 solutions.

>One thing that escapes "grug" is that achieving 1. often requires more sophistication than their magical club allows.

Which is irrelevant to grug's point, as he doesn't pretend to advocate achieving 1.

> Most well-intentioned "grug" developers will write software so simple that it becomes it's own form of complexity: a giant mud-ball of for-loops, while-loops, variable assignments, and other wonderful side effects. Instead of addressing complexity head-on with abstraction, "grug" will beat "galaxy brain" over the head.

Hah. Related, i became disgruntled with my previous favorite language for this reason. It promoted this style of development. Surface level simplicity built into the language, with minimal tools to actually manage the complexity that is inherent to the problem space you're working in.

Alright I'll bite :)

Python? Java? C? Assembly?

Not who you're replying to, but Tcl was that language for me. Dead-simple syntax, documentation, and behavior. I loved it. But also dead-simple tools. And if you wanted to do anything that wasn't a "solved problem" already, you had to write your own library.

When I switched my focus to Python, everything was more effort but I could also do _much_ more with it.

> And if you wanted to do anything that wasn't a "solved problem" already, you had to write your own library.

For which language is that statement not true ?

Their point is that the language they’re discussing requires you to write that library far more often than other languages, as a second order effect of few using it. But you know that already, you just want to make a semi-snarky reply instead, I guess.

No, I just sincerely don't believe that library availability indicates anything except library availability. It tells nothing about a language other than how man libraries are available.

Interesting - I found TCL to be very much the opposite, because it makes "meta" things so easy. Like, if you want to write your own control flow construct, you can go right ahead, it's like Lisp except that it's easier to write macros.

That would make for some interesting posts. "How X promotes wishful simplification," for various values of X. I imagine some common traits would be probably be "X makes simple things simple," with "it makes hard things look like simple ones." But there's also "X has defaults that Just Work" with "until they don't." I don't think I'd include assembly, C, or Java in those categories. Python? Maybe; "batteries included" is starting to break down. Go? Yep. Goroutines and channels usually work well for simple stuff, often for more complicated, and the core libraries follow that pattern. Model failures tend to be buried and solved when the service is turned off and back on. Rust? Don't think so; its schtik is making hard things into blockers. Web frameworks? Yes, oh very yes. Lisp? "Simple? Blink blink. Oh, you mean elegant."

My guess would be Go since I've heard that as a common complaint.

I was purposefully vague heh. Trying not to start a language flame.. i do that enough in the actual language threads lol.

I will say someone here was right though. :)

Python is this. Complex made easy (in Rich Hickeys terminology)

> a giant mud-ball of for-loops, while-loops, variable assignments, and other wonderful side effects.

Woah woah. This is literally the kind of complexity grug wants to avoid. Simple doesn't mean using the fewest language features or keywords, it means simple to read, follow along, reason about, and debug. Abstractions can aid in that (like using a third party library!), until they are imported/implemented aspirationally and thus often unnecessarily, and sometimes resulting in abstractions for abstractions, selective application of the abstractions vs fresh implementation, etc (...and thus AWS)

At no point does grug argue that you should stick to what you know, he just says you should trap your complexity in crystals ie literally suggesting to use the Good Abstractions, When They Make Sense.

^ sound convince and probably very big brain right at times

however!

beware apply advice put form data from web page into database with many layers abstraction not needed! grug see many time!

fear of looking dumb (FOLD) great danger in such conversations and beware!

You're focusing on coding style but it sounds more like hes talking about business requirement.

If we are talking about coding style, then the things you have identified as actually not simple are by definition not what the article favors.

I think the post actually touches on this? It doesn't say that you shouldn't abstract, just don't abstract too early.

I think of it as being more of a spectrum where abstractions get introduced as they're needed. The case where one has thousands of lines of imperative code that is terrible to debug and probably not unit tested is one where the correct abstractions were never introduced at the place and time they were appropriate. Getting this right is almost impossible, that's why we have refactoring practices.

As the idiom goes: Can’t see the forest for all the trees.

Sadly it’s often difficult to argue with someone using the complex=bad-club, because everybody knows complexity must be avoided - at all cost!!

What one should understand is the essential complexity of the problem, then design your solution around that, to control where the complexity goes and to not introduce accidental complexity. Blindly going complexity=bad will result in it popping up elsewhere later.

Many people as you say don’t even know what it means and use complex=bad for all kind of things they disagree with, including choice of tools or libraries. Using a library you don’t know is not complex, it is difficult, also something that should be avoided but that doesn’t make it the same thing. (Adding libraries and mixing competences can of course add complexity also, but let’s not get pedantic…)

That's the classic "only coder" mindset. Instead of being uncomfortable learning about software architecture, side effects or even pure functions, tries to solve every problem with for and while loops.

Sir, may I remind you that when you get down to it, your architecture and higher order functions are all made of branching and for and while loops, and branching and for and while loops are all made of goto.

And gotos are made of jump instructions which are made of NAND gates, transistors and silicon. Good abstraction is all about hiding the details that are not essential to the problem at hand.

I wanted to say jump instructions at first, but gotos are more hated and considered harmful in these quarters.

You hide the details but you never forget they’re there, lest they spoil and start smelling really bad.

(Also, apparently the MOV instruction on amd64 gives you Turing completeness as proved by that crazy compiler, so gotos may be made of MOVs sometimes, but meh)

A particular implementation might use those things, and I will meet it on its terms when I need to. But that's a mechanical convenience, nothing more. My top-level code is written for people to read, and only incidentally for machines to execute.

There is a lot of wishful thinking that goes into galaxy brain code. It assumes you know with almost complete certainty what rules are bound to change and where extension will be needed, that the next developer will be able to understand and respect the beautiful elegance of the design instead of hacking at it to achieve the desired result, and that the code will be long-lived enough to justify the initial investment. Grug coders aren't always inexperienced; many are actually repented architecture astronauts.

Then there's the Super Mario developer, I putta all 50,000,000 lines of code into one a fantastic pasta bowl file just for a you!

could you give an example of (1)?

Only because i have run into weirdness with printf calling malloc, back in the day. Even hello world makes me a little nervous about those claims.

But I'd love to see a sample with explicit assertions about the environment, so I could be sure there are obviously no errors.

Software in category (1) and (2) can be of any relative size.

1 could be 50,000 lines, 2 could be 500.

It will go away -- to other devs, who will awkwardly and painfully route around grug, adding loads of complexity. Not grug's complexity, though!

Reply on news.ycombinator.com