Ask HN: Do programmers generally enjoy some amount of complexity? | Hacker News Reader

Ask HN: Do programmers generally enjoy some amount of complexity?

26 pointskeepamovin57 comments

Occured to me that sometimes, simple enough is not good enough for us programmers. It seems that sometimes we might actually enjoy grappling with complex, even byzantine systems. Can anyone relate? I'm not saying this is a thing, not even for myself...but I suspect it might be, possible.

Some examples where I've encountered this possibility of things being perhaps more complicated than they need to be: ReactJS, Azure templates (and infra as code/config in general).

Could it be there's something satisfying that we need about working with complex systems? Do we "like the complexity" so much that we actually, sometimes, make more complexity than we need to, because that's more enjoyable?

Or is this shower-type thought safe to chuck in the receptacle of non-sensical but good sounding ideas (and other trendy intellectualism)?

edit: Not to side-track into naming, but just for those who might get stuck on terminology I use programmer here interchangeably with software engineer or developer (I know there might be differences, tho honestly not too sure what they are) - but basically those of us who work in this industry and produce code.

View on HN

This is the cross section of high intelligence and conscientiousness. Conscientiousness is the ability to perceive and abstract the world outside yourself and intelligence accounts of quantity of variables a person many consider simultaneously.

People who are both highly intelligent and have high conscientiousness are capable of perceiving many variables simultaneously and abstract them into refined simplified solutions. People with lower intelligence and high conscientiousness are still capable of perceiving these variables but will struggle more to put them together in a meaningful way necessary to achieve a polished result. It should be noted that conscientiousness is negatively correlated to intelligence at around -0.24 - 0.27, which is a big gap that explains why some lower intelligent people are unexpectedly more capable of high quality delivery than much higher intelligent peers.

People with low conscientiousness cannot refine complex qualities into simplified output. The cost is just too high irrespective of their intelligence. The result is sloppy output that just costs less to start over every time than to refine or extend. This will appear that these people tend to preference complexity, and in a manner of speaking that isn't completely wrong. It's not so much that they prefer complexity but that they are just incapable of expending the effort to build more meaningful systems.

Wow, this a great analysis. Never thought of it that way. Very interesting!

I think it's also true that the reality that software lives within or tries to model, always exceeds to capability of that model itself (at least for anything of significant complexity). Software has limits in other words to how good and bug free it can be. Reality will always win in the limit, and you get bugs and designs that deviate from a 'perfect' ideal.

What do you think?

"Conscientiousness is the ability to perceive and abstract the world outside yourself"

It sounds like you've confused consciousness with conscientiousness? First paragraph sounds like it's about consciousness. And second is about conscientiousness. Third is like a synthesis of both meanings?

EDIT Sorry, your comment actually makes a lot of sense, it's just that "Conscientiousness is the ability to perceive and abstract the world outside yourself" is a very weird definition, see [0] [1] - nothing about perception or abstraction.

[0] https://dictionary.cambridge.org/dictionary/english/conscien...

[1] https://en.wikipedia.org/wiki/Conscientiousness

Nice word, isn't it?

I agree that the parent poster's usage is unconventional, but I quite naturally interpret it as: conscience dictates that sloppy thinking, disingenuous arguments, and self-delusion are to be avoided; that one has a moral duty to reason correctly on the basis of available perceptions and knowledge, and pursue true conclusions even if it's to one's own detriment.

Stands to reason that being inherently motivated towards that sort of thing improves one's capacity for abstract thought, and thus effective action in complex domains.

In the computer metaphor, intelligence would be how much RAM you have, while conscientiousness would be how often your CPU has a glitch and performs an incorrect computation due to "overclocking", i.e. burdening your mind with self-serving yet ultimately unproductive cognitions.

No. Consciousness is the waking self, the ability to perceive at all in a sober and non-dreaming fashion. With regard to consciousness the ability to perceive at all is sufficient regardless of whether that perceptive ability is limited to yourself, a narrow subject, or details around you. The oppose is unconsciousness, which is typically thought of as the sleeping mind.

Conscientiousness is more distinct, a subset. It is the fine detail by which a person can perceive dates, times, and schedules. Other facets of conscientiousness include social intelligence, empathy, organizational capacity, cleanliness/orderliness, details about nature, comparisons and measures, and more.

Low conscientious people struggle to think beyond themselves and struggle to find the energy to perform tasks not considered immediately pleasurable or self-rewarding. That is what makes the costs of abstract task completion so very expensive, because the effort required is beyond consideration. For example a simple task like cleaning your room or waking early without an alarm clock is an impossibility for some people and for others it is as effortless as getting dressed. For people who struggle with these they appear more comfortable with complexity or disorderly, which is a false conclusion. These disordered result may be very stressful and perhaps over stimulating, but the effort to correct for it is too expensive to resolve. That is why some programmers can seem to build anything and refactor large applications in a few hours while other developers struggle to merely formulate a starting idea in that time and an equivalent refactor would take weeks to accomplish.

Extremely low conscientiousness is linked to a variety of mental and behavioral health disorders because it is so completely detrimental to standards of living and real world performance. Extremely high conscientiousness is linked to perfectionism, OCD, and anxiety in a world of disordered imperfections.

I consider myself to be on the lower conscientiousness side of the spectrum. For others who are like me, what has worked best to improve myself in this area are these two things:

1) develop empathy and compassion for yourself. celebrate wins, and analyze losses.

2) make small, concrete steps to simplify your environment. as the above commentor mentioned, task completion is expensive. however, some things decrease the cost of task completion or increase your available energy. among these are: habits, good sleep, consistent exercise, a decluttered environment, proper nutrition.

together these form a virtious cycle, improving your capacity to make meaningful decisions by either increasing the energy you have available to you, or by decreasing energy drain from other areas.

finally, remember that some things are not worth thinking too much about.

>conscientiousness include social intelligence, empathy, organizational capacity, cleanliness/orderliness, details about nature, comparisons and measures

I'm not sure what "comparisons and measures" means but the rest checks out for me, I had no idea these things are corelated. I used to think about conscientiousness as "behavoiral" lack of discipline, but it totally makes sense that it affects perception of reality deeply.

People with exceptionally low conscientiousness struggle to compare two or more things because they lack the ability to externalize such comparisons from themselves. Instead each item will be considered on a value system in relation to the person and then those relationships are what’s compared. That is both biased and logically absurd.

The same logic applies to measurements. It simply costs so much less to just guess at a measure that is self pleasing that it isn’t cost effective to use any external tool to form an objective result.

Very interesting take that is probably true in my experience. (And not to say I doubt you at all but would be interested in a citation for that correlation.)

I think also that bright people like to solve problems, and not all bright people consider complexity a problem - and indeed, whether the intent is consciously there or not, it provides future opportunities to solve tricky problems!

In the context of scrum, with short sprints aimed at delivering immediate business value, it's challenging to make simplifications that have positive value in the long term, especially when you're unsure at the outset whether such simplifications are even possible. Meanwhile if you got the desired outputs from the specified inputs, you had a fun enough time conquering the complexity and you can get a pat on the back and move on to the next challenge.

My theory is that an org works best with both types of personalities, and if it knows what's good for it (especially if it's bought into scrum) knows who its best refiners and simplifiers are and lets them have at that kind of work while others concentrate on the immediate delivery pressures (usually more junior engineers but sometimes also just career specialists in fast delivery whatever the complexity cost).

I have respect for all these people as long as they have respect for each other.

I really love this answer and I'm definitely stealing some of it for not only my own inspiration but something to consider when hiring.

I've never heard of this concept before, which seems strange because I feel it applies so much to be a natural sentiment after reading it. I've had the 'feeling' something like this was at play but never knew how to define it. I'm curious, are there any specific books or lecture or any material that brought you to that clarity or is it just experience ?

I absolutely hate complexity, and will work very hard to avoid it. So hard, in fact, that it can seem that my productivity is zero for long periods of time. I will think for days or rewrite the same thing several times if the code is important enough and then—sometimes—out plops some small piece of code that does the right thing.

Most people I have worked with get very impatient with me. I tend to work best with things that actually have to work all the time. I have written code that sits forgotten doing its thing for many years, while more visible contributors get kudos for solving disasters on weekends. Not bitter, of course ;)

Hahah - this is a cool way to work. Prepare in secret, strike like thunder! It's funny the rewards for 'solving problems we created' (i imagine there are whole industries that work this way). Sometimes when I see the "next version" of popular tools/frameworks dealing with the fallout from enthusiastically adopted shiny abstractions of prior versions, I think of this :)

Seems to me the meta version of this is there exist entire jobs and careers to support technologies that are over complex, badly written, poorly documented, need constant security updates, patches etc, keep changing the way they work, need an enormous amount of effort to keep running, vendor gets bought out or does a pointless redesign, while something simple written by you or I just carries on working quietly and gets no attention. This is a tragedy of tech. I fully believe in the KISS (Keep It Simple, Stupid) principle. The world could spend way less on tech if we all followed that. Its not even just tech - e:g Henry vacuum cleaners are used on filthy construction sites and seem to last forever while more expensive fashionable ones break in 5 mins. Simplicity being the key difference.

You say complexity, I say lucrative support contract.

Lol thank you for this.

I think it's not so much that people like complexity itself, but it's a result of punting on decisions and failure to make tradeoffs and design decisions.

Instead of writing something that does one thing and does it well, people write something that can run in four different modes, and is configurable via a yaml file, and can load plugins to change the behavior, and isn't a finished/standalone product but a "framework" that can be configured into a product...

A lot of it just comes from punting on bulletproof designs too, like some code that works 99% of the time, but every now and then it fails due to some problem like a race condition or a deadlock. So... instead of fixing it, someone wraps that layer with something that will retry N times, and N is configurable, and then every now and then it fails N times in a row so even with the retries it fails, and then people are like "we should increase the N configuration parameter!", and then other times it just sits there retrying all day and people want to know why it's stuck, turns out its still retrying "oh no, we should decrease N, so it fails early and someone can fix it, and we should add monitoring and telemetry and alerting so we can see when its in a retry loop!"

In this scenario, there is no good value for N, so it just shouldn't exist, the right answer is to fix race condition/deadlock so it just works every time.

For me, I enjoy solving complex problems as simply as possible. Also, in my career I've worked with devs for whom the complexity of their solution is a badge of honour.

I like that too, but it's pretty hard to do. The best design is when there's no more to take away. Nice ideal, but sometimes I don't have time for aesthetics, I need to hit other aesthetics or ship ha. I try not to take it so far into complex as badge of honor tho - it's mainly about self-respect - I'm gonna be maintaining that code (or someone else is) - I want to be kind to myself.

I believe complexity is nothing that can be created or destroyed, but just moved around. Sure, you can create artificial complexity by making things harder than they need to be, but that just the complexity of the side effects you added accidentally. When you reduce the solution down to what is needed to solve the actual problem at hand, the complexity required to do so will always be the same. Now, you can hide that complexity behind an abstraction; it’s still there, but hidden behind a trade-off: you placed a cover over the gears, made it look nice and clean, but if you have to change gears, you’ll need to remove that cover first, and possibly make a new one afterwards. The overall system complexity stays the same.

Now, I think what most engineers enjoy is the act of slicing the complexity in neat boxes, and shift them around, until it feels balanced. It’s a very similar task as building a cairn from pebbles, actually, just on an intellectual level. You get the joy out of a feeling of accomplishment when the stack balances out.

That's a good attitude too. Maybe if you are employing a 'bad programmer' on your team you are just moving the complexity in their own brain/worldview into your team/codebase hahaha! :)

I agree with you that there's unavoidable complexity and there's a limit. Let's assume we can even get to that clarity, but it's achievable to get there - and once you get there you realize that we just have to dive in. There's no way around it. Even so I'm always looking for that simple-as-possible, first-path-to-poc way to do it.

Another thing you said is interesting too: the balance of complexity (almost aesthetic that arranging isn't it?). That's very clever and I that describes the 'dark side' of 'good designs' that are actually not useful, even tho they may appear elegant.

This has produced a lot of really interesting answers. I feel condifent saying that programmers in general definitely work with, think about, and are intimately invovled with complexity. I think because we are (in a down to earth not grandiose way) at the coalface of programming reality, and reality is complex.

Solving complex problems with simple systems is really really difficult.

Most people can't do it.

Even worse, most businesses are actively preventing it with the way they want their teams to work. How many people reading this can honestly say they get time to address technical debt or rework things to fit a more coherent view of the overall pattern that has developed with time and fixes and added features? How many developers can truly say they get to choose the best technology to do their job? Do you actually use the languages and tools that fit your domain best?

Almost none of us, right?

Also some of what you cite is fairly early-stage tech. IaC is still such a new concept that it's not surprising nobody's figured out how to make something that isn't incredibly annoying yet. Terraform, Bicep, whatever, it's all pretty raw and hideous still.

It's not so much that we like complex and messy tools and solutions, it's that we don't - for various reasons - have a choice.

Hahaha this is a great answer, and enjoyable to read. I heartily concur with your idea that most businesses are actively preventing it with the way they want their teams to work

I think this is what early Google did well. Engineers were unleashed to do their best work - or so I've heard, maybe it's just a nostalgic myth seen through the gilded glasses of past recollections.

I also think it's such an important problem to solve, and low hanging fruit! Otherwise you are impeding your team's ability to solve problems and retarding your ability to compete with other orgs. Just create the conditions to ensure the 'production line' is optimized. I don't just mean cranking out code, I mean optimizing for what the goal really is, and what's going to help you achieve it, in that Fordian sense.

Also, raw and hideous lol! Perfect description hahaha.

What we enjoy is the management of complexity. Not only is "complexity" neither good nor bad by itself, not only is the same true of "simplicity", but no two people will even agree on what these things are.

Spoken like a true programmer, who knows the trade-offs of all the things! :)

> Do programmers generally enjoy some amount of complexity?

Yes, to their own (and the team's) detriment.

I really hate complex/clever/dense code. Just about as much as I hate "Look I found a want to look through this and write it once instead of mapping every property 1-by-1"!

Every, single, time I've seen that "shortcut code" it's turned from:

    foreach($item as $key => $value) {
        $newItem->$key = $value
    }

Into

    foreach($item as $key => $value) {
        if(/*special case 1*/) {
            // Special Logic
        } else if(/*special case 2*/) {
            // Special Logic
        } else if(/*special case 3*/) {
            // Special Logic
        } else if(/*special case 4*/) {
            // Special Logic
        } else {
            $newItem->$key = $value
        }
    }

In record time.

Just doing:

    $newItem->keyOne = $item->keyOne;
    $newItem->keyTwo = $item->keyTwo;
    $newItem->keyThree = $item->keythree;

Is much more maintainable and gives you unlimited "hooks" if you need special logic for a given key.

Note: this is a contrived example as the keys are the same, most cases I've seen this in dealing with mapping DTOs (maybe going from snake_case to camelCase) in/out of a system, a use-case where I 100% do not want "magic".

I've moved from code-golfing and being as clever as I can to loving boilerplate and verbose code.

I don't think tyhat's a contribed example. I think tha'ts a really good example that makes super clear what pointless abstraction is in a perfect pithy little package.

> Or is this shower-type thought safe to chuck in the receptacle of non-sensical but good sounding ideas

Regardless of veracity, having an answer is reductive beyond the point of utility.

Like, yeah, it's reasonable to infer that someone who has chosen a career in programming might enjoy a nice game of Factorio. But beyond that, what is actionable or predictive about knowing?

It's worse than that. People will start making the hidden little assumption that the complexity of a deliverable is strongly correlated with the practitioner's propensity for complexity.

It's just blaming developers for bad software with extra steps. Instead of considering the dozen bigger factors that lead to shipping imperfection, we just have another reason to pull out the tired old excuse of blaming the programmers and their broken nerd brains that focused too hard on nerding instead of delivering business value.

No I totally see your point I think people are too unfair to programmers in general - seeing software as a cost center and not appreciating the complexity or difficulty inherent in it - and I get how even the question suggests a kind of negative stereotype that might damage people's interactions with non-programmers, for sure.

I came at it from a more positive viewpoint - not of the connection with complexity being a bad thing, but just exploring the dynamic and wondering what it means for us. I think a lot of the answers are really interesting tales born from that experience of working with it day to day, and I think wrestling with the complexity of the world is central to what we do. Celebrate our heroic victory, dance with the complexity of the world! :)

Odd if nobody has mentioned yet how dominant frameworks such as React are usually hand-me-downs from market incumbents. I view them primarily as elaborate false abstractions; heavily astroturfed, intentionally suboptimal solutions for the sake of disempowering workers and creating jerbs, thus precluding emergent competition.

A junior would get a nice sense of mastery from figuring what the hell the framework authors must've been thinking when they designed it ass-backwards, but in the long run would burn out and become unwilling to learn the (usually simpler) layer underneath the abstraction. While a senior who's become wise to the racket would have to contend with the next crop of junes normalizing an even more nonsensical tool for accomplishing even less with even more because that's what the influencers influenced them to think.

I totally have the same view. At least I did when I was focused on them. Now I barely think about them at all. Something pernicious about it, isn't it? The deliberate over-engineering of false complexity to bog down potential competitors! Shed a tear for the billions of human hours wasted by millions of startups on stupid ideas in these frameworks!

A junior would get a nice sense of mastery from figuring what the hell the framework authors must've been thinking when they designed it ass-backwards lol, totally! But I also feel like it would ruin that junior's brain! I always avoided them, luckily my aesthetic intuition totally warned me off kept me miles away. I am not poisoning myself with that hahaha! :)

Hopefully no acolytes encounter these comments - the medicine will be too bitter for them. Your answer is the vibe of a whole era (2010s shiny engineering), thank you!

>aesthetic intuition

The quality. Thank you for keeping the fire burning!

Yes, I think there are software producers (there's a new title to throw in there :D ) that over-complicate solutions maybe naturally because of how they are wired or intentionally because of some need to be better than others or they see complicated things in the industry and feel like using them.

A lot of complexity is defensive, that comes from limited information on how the code will run.

For example if you’re asked to write a server with a public handler with an access to an important db, you may think of rate limiting it immediately.

Or if you have to compare two slices of a numeric stream to find out which numbers are “new”, you’ll spend half a day adapting and testing a complex dynamic programming routine, cause you don’t know if O(n^3)-ing it is fine cause n never exceeds 10 irl.

Programmers receive understated tasks all the time. Simplest solutions tend to fail and create Friday night recovery, so they learn to foresee the obvious and forecomplicate things to turn Friday nights into Friday evenings.

You don’t know what to take away. You may not even be sure how you will use your own system yourself.

That's an interesting take! Complexity is necessary defensiveness. Exactly - a lot of it can be that. Like well-designed complexity. Complexity because of the model. Whereas maybe we can say 'bad complexity' is complexity in spite of the model.

I guess this is exacerbated in waterfall type designs or processes without iterative feedback loops. I like to code with tight loops to know the behavior and adapt.

I think that two things are happening here. First, we need to acknowledge that simplicity is hard. Requirements change, and making things as simple as they can be (but not simpler) requires a lot of effort.

Second, programmers love spotting things that can be optimized for some definition of what that means - whether it’s faster, more flexible, more secure, more future-proof, more reliable, etc. We don’t like leaving low hanging fruit. The only natural limit to these optimizations is the complexity they add, and the idea that some improvements are not “worth it”. As a direct consequence, we end up with the most complex solutions we can still reason about.

I actually love delving into legacy systems, trying to understand Design decisions, figuring out the incongruencies etc.

It gets considerably less fun if you are under time pressure or have any kind of expectation to make effective changes.

For me, no. My motto is "as simple as it can be, as complex as it needs to be". I don't want to build overly complex systems for the sake of building it.

What I enjoy more is seeing things work. Without any luck or non-determinism in my code. "I press button, I get coffee" type of predictability. Boring is good. Boring is reliable.

I also enjoy when my code works fast. Without a mountain of hidden dependencies, absurd amount of resource usage, and "please get a coffee, and come back when it finishes" length of waiting.

Things should be snappy, light, and simple. And preferably GPLv3 licensed.

> Occured to me that sometimes, simple enough is not good enough for us programmers

There's a beautiful article recently linked about naming conventions, there's also another about "The worst developer". Incredible reads.

Simple is good, but it depends on context, it's an absolute certainty we like reducing problems to solvable gradients. This is a good thing.

We all love problems, solving them is 1 challenge, solving them beautifully is another #2 thing.. Never making those problems is an architect thing. We all grow to evolve but learn from eachother.

As a classic, or "physics based" engineer, I'd argue that software engineering is pure complexity extract. Take away all the environmental variables, make everything a perfect binary 1 or 0, and then do engineering in that space. The only thing that can really give value in that space is complexity.

I've worked on some pretty complex electronics, but even the most insanely engineered projects don't even begin the scratch the surface of the complexity in an average commercial grade software program.

So interesting, thank you! Yes software is so complex I don't know how we keep it in our heads! It's why tooling (idea tooling especially) is important. And experience, so you can absorb-away a lot of 'muscle memory' and conceptual heuristics and quickly make the right choices (even if the right choice in the moment is re-evaluating your previous ones and being open to alternatives!!!)

Speaking for myself, I am impressed by something that is necessarily complex in order to accomplish some difficult task, and dislike needless complexity that could be simpler. A good example of necessary complexity would be this post from yesterday https://news.ycombinator.com/item?id=43448457. Another example would be how the A-10 had to be fueled right before takeoff because it would leak fuel until at top speed.

> Another example would be how the A-10 had to be fueled right before takeoff because it would leak fuel until at top speed.

I think you mean SR-71? The A-10 is very hardened for close air support. It even has self sealing fuel tanks

https://www.af.mil/About-Us/Fact-Sheets/Display/Article/1044...

I did; thank you for the correction.

I kind of like my software as I like my cars: Simple, and easy to troubleshoot.

The more predictable behavior, the better...so choose your tools and methodology wisely. The older I've become, the more I appreciate doing something that might be hard and challenging / lots of up-front costs during development, if it yields a robust system which is easier to fix, should something happen. So I will go for battle tested solutions, over the novel ones.

Yeah I like this idea, too. Testable, reliable -- and simple as possible - more important the more experience I have. I find myself leaning and advocating for the simpler of options, the one that gets to prototype, or feedback or PoC, sooner, knowing that more can be added later. Broad strokes, then fine strokes.

One silly but I guess not bad "test" I had for a technology or choices was: If I had to debug this for a critical problem at 3AM coming home from something else and am intoxicated, can I do it? If it doesn't pass that test (the "can I debug this at 3AM drunk" test), if it's too complex, I'm wary of it! Hahaha :)

Few things rival the satisfaction of solving an extremely complex problem with numerous constraints by producing an elegant, simple solution. When your best thinking, experience, knowledge—and often collaboration—lead to such clarity, the feeling must be similar to what mathematicians like Euler experienced when discovering their most beautiful proofs.

Simplicity is the ultimate sophistication.

“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” ― Antoine de Saint-Exupéry, Airman's Odyssey

Both of the two ancestor comments are perfectly stated.

It's about finding a simple (enough) way through a (seemingly) complicated system.

YES! Clearly an experienced developer in the house here. Your effortless focus on the key thing reveals it. You cannot hide!

Generally, no. Some folks do, some don’t. It’s uncommon to find anything subjective that generalizes to an entire population of folks with knowledge of a specific widespread skill.

There are two kinds of problem solver: The ones that think that a complex problem requires a complex solution and those who try to solve complex problems with elegant solutions.

The first group transfers all the complexity directly to the solution, the latter tries to eradicate complexity by trying to simplify the problem first.

I found that engineers tend to be usually in the first group. Designers (especially UX's) tend to form the second group.

Example:

This is (popular) software made by the first group: https://www.bulkrenameutility.co.uk/assets/img-bru/mainscr.p...

This is essentially the solution at the same problem, made by the second group: https://cdn.osxdaily.com/wp-content/uploads/2015/05/choose-b...