Hacker News Reader

Comment by deathanatos | Hacker News Reader

I wonder if the author has read Out of the Tar Pit[1]¹. See, in particular, section 6.

Essentially, the author is, I think, arguing about what the paper calls "Essential complexity", complexity inherent to the problem one is trying to solve. And with that, I agree.

I think the author should acknowledge accidental complexity (or provide some argument as to why that must live somewhere), and I think a lot of comments here on HN are pointing out the fact that accidental complexity exists, and doesn't have to live somewhere. But my guess is that that's not what the author is saying, and that the author is only arguing about essential complexity.

[1]: https://github.com/papers-we-love/papers-we-love/blob/master...

¹I personally found this paper somewhat mixed. The definitions of complexity are what make it worth reading. Its conclusion of "functional programming languages will fix all the woes" I think is not practical.

View on HN

Yes, there's definitely a kind of complexity that _doesn't_ stem from the problem domain, and it can often be eliminated.

Stuff like over-abstracting, insane default values, duplicated state that needs to be synchronized (e.g in React components) or just overly-repetitive code, for example.

Manual memory management is accidental complexity unless very strict memory usage rules are part of the specification.

Some kinds of type systems similarly lead to accidental complexity, for example in the form of repetitive code because the type system makes sufficient abstraction impossible. It's possible for some kinds of requirements to mandate very concrete types with no room for abstraction in some parts of a program, but even then the compiler can do most of the work with making sufficiently abstract code concrete and performant.

> Yes, there's definitely a kind of complexity that _doesn't_ stem from the problem domain, and it can often be eliminated.

Yes, it can be eliminated, but only to some extent. Programming languages add a baseline of complexity by themselves that _can't_ be removed: programming languages are not infinitely flexible so there will always be problem characteristics (invariants, data structures, etc) that will not be easily expressible. Those are new sources of complexity.

Think how Java simplifies manual memory management when compared with C or C++. Or how the Rust borrow-checker provably prevents a whole class of bugs. But these are all tradeoffs, both Java and Rust are ill-suited to express other problems.

I notice that there's a kind of complexity that seems to straddle the bounds, related to the way it easily turns into effort-reallocation politics. Would it be useful to consider “coordinative complexity” a separate type that's intermediary on the essential–accidental axis but also has other properties?

> Would it be useful to consider “coordinative complexity” a separate type that's intermediary on the essential–accidental axis but also has other properties?

The complexity of getting different pieces to talk to each other? I can see how that's only conditionally "essential" in that, if you have enough control over the environment, you can make the various distributed components as similar as possible, never deal with version skew, and never deal with varying interpretations of the same protocol standard.

OTOH, if you're writing a web page, and you have to make that page useful across multiple browser types and versions, some way of dealing with the vagaries of different browser implementations becomes an essential complexity: You either push it into a library or framework or you handle it in the main codebase, but something has to know what MSIE can and cannot handle.

Pretty much yes, though I was angling toward coordination difficulties between people creating complexity which is “accidental” from the perspective of the whole system but “essential” to individual actors given their incentives and what they can rely on, either for purely social-maneuvering reasons or for other reasons as well. And more specifically, how this happens even when the accidental complexity isn't really in line with the group's values or with the values of the individual actors. The way that this grounds out in a technical sense is scenarios like the one you describe, so I think we're talking about the same broad set of phenomena.

Further, there's different levels and causes of “coordination didn't happen”, such as “we tried, but got confused and didn't have time to understand each other properly” or “actively sabotaging everyone else's ability to deal with the aggregate system so that I can get ahead”… and somewhere in there is one that stands out to me as specifically interacting with technical complexity, along the lines of “I want my piece to not have to be the one that changes and/or takes on other costs”. Which is part of the social dynamic of moving externally-essential complexity around within the aggregate system and potentially creating ripples of accidental complexity along the way.

(Whether and how this also happens when the information systems in question are human minds rather than software components might be relevant but also an exercise for the reader…)

Reply on news.ycombinator.com