Hacker News Reader

The question is: is this thing inherently complex or is it more complicated than it needs to be?

In software there are many things that are more complex than they need to be.

Software development is an additive practice, where opinions and ideas are usually added to a big-ole-shipping-ball of software. I think this is where most complexity comes from.

Sometimes it can be a subtractive practice, where a person can delete / simplify, but this happens much less than it should.

Same with security, unfortunately.

Ironicaly, complexity is the enemy of security (as I’ve heard quoted a few times)

Zawinski's Law

> “Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.”

The real problem with software is that it's hard to change your mind once things are built on top of it. The real solution is to find a way that makes it easy to do this.

Complexity is one of the manifestations of this problem, but that doesn't mean that complexity cannot arise due to other reasons (i.e. essential vs incidental).

The answer is more along the line of better refactoring (or programming) tools, stronger type systems, better cross-program and whole system static checking etc.

I think that all else being equal, simplicity is to be preferred over complexity. You're pointing out that all else is never equal; ok, cool; but it's hard to imagine anyone finding value in deliberately complicating things. Simplicity seems to be an inherently valuable trait in system design/maintenance. Can you oversimplify? Sure. Does that justify not striving to reduce to the simplest functionally complete, scalable, and maintainable design? In my opinion, no.

The parents point is that there is a baseline of complexity which can’t be factored out due to the social and economic conditions under which software is designed and constructed.

> but it's hard to imagine anyone finding value in deliberately complicating things

Except for that special guru who writes The Framework for the company, because why not a Forth-based DSL for everyone to work in? Overblown, I know, but there are kitchen sink types, or those who just don't know that there's an easier way, and other reasons to wind up with too many tools or layers or libraries.

(I've been guilty of not knowing easier ways, so...)

even if this is true (and it definitely can be) this would fall into the accidental category of complexity and be a magnitude less impactful than the essential fact that "building software is hard because it is".

I'll try to make a parallel that in my opinion explains the thought behind the OPs argument.

In the early days people usually lived close to water bodies (rivers, lakes, etc.) because the closer you lived to the water the better your society thrived (water being essential for live, agriculture, transportation, etc). But there is a limit where if you go too far (i.e. into the water) the benefits evaporate possibly reversing the benefits (building houses into the water adds complexity, flooding, etc).

So there is probably a (global?) optimum that is neither too complex nor too simplistic

My points are:

1. Even software with no "accidental" complexity is necessarily complex (in fact, that's why it's useful).

2. If the manifesto's point is that we should stop over-complicating things, then wouldn't it better to first examine whether, and if so why, things are overcomplicated now, given that the previous generation also went on a crusade to abolish (over)complication? If the point is to declare "don't overcomplicate!" then, if you believe things are overcomplicated now, why is that declaration useful if it's clearly not effective?

Yep, now that we have lots of version controls histories publically available there is no excuse for this.

> Every new generation of programmers starts a crusade to simplify without first trying to understand why we're not "simple" despite the previous generation having done the same.

This 100%.

I'd also add that, often times, attempts at simplifying end up having the opposite effect and actually increase complexity because, as you say, the reasons for the current complexity haven't been explored/understood.

> one possible explanation could be that without a constant crusade for simplicity we'd be in a far worse place; the "permanent revolution" explanation.

I think the explanation is similar, but slightly more complex, than this.

I don't think we'd necessarily be in a worse place without crusades like this, because I don't really believe they have any impact on general trends in software. But that's not the same as having no impact or value.

Rather, I think these crusades are important for their niche influence; without them, we would only have the overcomplicated mainstream, without the minority of simpler options, that will never come to dominance but, hopefully, with the help of advocacy like this, will continue to have some small following.

> there needs to be a constant struggle not to over-complicate, but it should be done with the understanding that over-simplification is just as problematic

While I agree with the sentiment, the latter seems to be less of a problem in practice, no?

There is also very much accidental complexity in software. Mainly the plethora of OSes, programming languages, build systems, package managers and what not. For maximum simplicity there should exist only one of the things I mentioned in the previous sentence. Whenever one runs into one of these things and one happens to not be very familiar with it, one is in big trouble, and not the kind of trouble that, at least I, enjoy. Also, for simplicity there should not be the difference between software that runs on local machines and software that runs on the web.

The reason this plethora exists may well the be crusade that you mention. Instead of solving the problems with the current programming language, let us just invent a new one instead. The former thing is much harder to pull of, of course but should limit the amount of complexity in the long run.

Every generation simplify what used to be really hard. But then people use the new simple stuff to build solutions to a lot more complex issues than was feasible before, until they reach the complexity roof of the new abstractions.

Still, I think that the kind of people who become programmers like complexity, and without a specific concern for simplicity will create over-complex systems.

> Of course, there needs to be a constant struggle not to over-complicate, but it should be done with the understanding that over-simplification is just as problematic.

I would like to differentiate simplification, aka "dumbing things down", from purification, which is finding the right frame of reference to enable a more compact representation of the necessary complexity. Purification is always highly desirable (unless it's just a facade used to make oversimplification more appealing).

The problem with purification is that you usually don't have the time to ruminate on your problem space for months.

Yeah what you said.

There probably are exceptions. Software can sometimes be complex for no good reason except maybe its architects didn’t have a good plan and folks just added more bad code on top of existing bad code. I’m just not sure if that’s as common as the crusaders want it to be, and when folks say they want to simplify, I think that most of the time it’s because they don’t understand what they’re getting themselves into.

The universe is built out of simple things but there is complexity behind how those simple things are constructed and can interact.

Abstractions are a way to create contexts where contexts might not apparently exists and the main objective of these contexts is to provide simple models at every contextual layer to understand the infinitely complex environment where everything exists.

Not sure which generation you're in and which generation you'd (relatively) consider Alan Key in, but here is a much more principled approach to what the OP is trying to do: https://youtu.be/NdSD07U5uBs

The TLDW is that Kay thinks we haven't discovered all the fundamental rules of computing, a la Maxwells Equations for physics. If you are able to figure these out, you can substantially cut down your accidental complexity.

FWIW, my unscientific view is that achieving simplicity is really hard and doesn't scale people-wise. Instead of C99 without a bunch of stuff, I'd probably use Scheme to create very dense (lots of bespoke abstractions and idioms) but simple software. That approach works reasonably well, but try onboarding a new developer!

> Instead of C99 without a bunch of stuff, I'd probably use Scheme to create very dense (lots of bespoke abstractions and idioms) but simple software. That approach works reasonably well, but try onboarding a new developer!

I understand what you're getting at, but I'd argue that what you're suggesting isn't actually simple by any means.

I've yet to watch Kay's talk, but I take issue with the fundamental claim that rules necessarily lead to "simplicity". If there is one thing computer science tells us is that this is not the case. For example, orbit equations are very simple, yet if you put more than 2 objects in gravitational interaction, their behavior is completely unpredictable (and very complex) even though the rules that govern it are simple. I'm personally interested in (and practice) formal methods for software verification, and I saw these nice slides in a talk [1] by a formal methods researcher that say:

> Computers have been invented to surprise us

> If we knew what computers do, we would not use them, and we would not have built any.

This means that computers are most useful precisely where the complexity of the problem is beyond our efficient reasoning powers. Like Brooks, I think that our accidental complexity is already quite low (no more than 50%), and while it's always good to decrease it, what we're left with will still be very complex -- that's why it's useful.

[1]: https://shemesh.larc.nasa.gov/NFM2018/talks/Dowek.pdf

If you don't like videos, you can also look up his papers from STEPS circa 2010.

I think the 3 body problem is indicative of what Kay says. One simple rule produces very rich behavior. Why not have something like that for your network stack?

> Why not have something like that for your network stack?

In a way, it is like that already. After all, all computation can be described as a repeated application of, say, the very simple reduction rules of the lambda calculus. How does that help? Does the fact we know what the gravity equation help us predict the system's behavior (e.g. to ensure that some bad state is never reached)?

You bring up a good point, but do you have any specific arguments about why you think the authors of this manifesto have not tried to understand the complexity we currently have?