Hacker News Reader

That's funny because I have become, and observe the exact opposite.

The notion of trying to make things more complex than they need to be I find deeply problematic.

Junior engineers often end up devising solutions which are in fact too complex as they try to over engineer everything.

The more experience I have, the more I accept that code is complexity and cost and that it should be avoided.

Always take the easy path unless there are very good reasons not to ... because you'll never know where you need to actually pivot when you get there.

To your point (not more complex than they actually are) - yes, there is sometimes inherent complexity in the problem space that can't be avoided - but that doesn't mean the solution actually needs to be complicated. Domain complexity might possibly be well encapsulated, leaving the rest of the system fairly straight forward.

> Junior engineers often end up devising solutions which are in fact too complex as they try to over engineer everything.

> The more experience I have, the more I accept that code is complexity and cost and that it should be avoided.

At a previous company I worked for I proposed a solution which was considered to be too "complex" by the VP R&D. So he and another "Senior" turned around and rewrote it into something more "simple."

This "simplicity" slowly turned into a constantly patched clusterfuck of a mess and it stripped off a dozen different ways of customization. The "simplicity" was produced as a consequence of both of the Senior engineer's ignorance towards to the tools (language, frameworks) we were using and they just went with their gut experience on it.

I'm somewhat different to you I guess. As things go on, I realise that simplicity is usually a hallmark of developers who don't really understand the tools they're working with and can't really foresee where the business would be going with that feature in 12 to 24 months.

It's a symptom of "Jira culture," where people develop things exclusively to satisfy their ticket and ignore the bigger picture of what they're actually trying to achieve.

This is not to discount people over-engineering things, but over-engineering is very different from complexity.

Looks like the two of you have a different worldview about what 'simplicity' means.

It really nice of you to notice. Some crusaders still think it's about complex vs simple.

This is because people don't realise that there are two axes on the simplicity/complexity graph, not one. So they're operating on different definitions of 'simple'.

You can make the development experience simpler, and/or you can make the software simpler.

Usually a change to one will have an inverse affect on the other. Bringing in a library will almost always make dev simpler (unless the API is more complex than whatever it's abstracting, which only seems to happen in JS land), but it's also guaranteed to make your software more complex (unless it does exactly what you need with no config, and nothing more).

I tend to observe that junior engineers simply misplace the level of complexity that a system needs in order to be as simple as possible. Meaning, I've seen errors to the side of being too simple, and errors to the side of being too complex; the unifying theory here is that its a mis-estimation of "necessary complexity".

Two examples:

In designing an email templating system, I've seen junior engineers devise really complex class hierarchies, with super classes that take in type generics for their super() constructor, three levels deep, resulting in hundreds of lines of code. When pressed, the reasoning was "there are parts of the email body we want to be the same and parts that are different, so the class hierarchies map to those parts that are similar vs different." It turns out, this service was only sending 3 different email "types" for the foreseeable future, and the only similarity between them was a copyright notice at the bottom, a banner at the top, and a body.

But, a counterpoint: For a long while, on some node.js backend API we were writing, much of the core business logic was implemented as simple functions across hundreds of files. Worked great when we had four API routes. Now, with hundreds, its impossible to find anything, because there's no structure, patterns, or organization. So, we needed someone to step in and introduce rules and structure in order to make the system more simple, and our job would have been easier if we'd had the foresight to see that necessary complexity coming.

Surely good examples, but the later may have been the right path: start simple with what is necessary. When there is a need for a framework or some layer of complexity - add it then. But maybe not before.

And #1 is funny because we've all done something like that!

> In designing an email templating system, I've seen junior engineers devise really complex class hierarchies, with super classes that take in type generics for their super() constructor, three levels deep, resulting in hundreds of lines of code. When pressed, the reasoning was "there are parts of the email body we want to be the same and parts that are different, so the class hierarchies map to those parts that are similar vs different." It turns out, this service was only sending 3 different email "types" for the foreseeable future, and the only similarity between them was a copyright notice at the bottom, a banner at the top, and a body.

Was there a discussion about high-level design of the templating system at all before the junior started banging out code? This feels more like a failure of the senior/lead.

> But, a counterpoint: For a long while, on some node.js backend API we were writing, much of the core business logic was implemented as simple functions across hundreds of files. Worked great when we had four API routes. Now, with hundreds, its impossible to find anything, because there's no structure, patterns, or organization. So, we needed someone to step in and introduce rules and structure in order to make the system more simple, and our job would have been easier if we'd had the foresight to see that necessary complexity coming.

This feels like a false dichotomy to me. Your system existed in a state somewhere between between 4 routes and hundreds, didn't it? At the point something starts being painful is when it should be addressed, imo. That was probably somewhere in the teens of routes would be my guess. At that point the system is still small and understandable enough to make a large architectural change, but it's big enough that you can be reasonably certain you're making the correct large architectural change.

> Was there a discussion about high-level design of the templating system at all before the junior started banging out code? This feels more like a failure of the senior/lead.

There was not. This is a startup I'm referencing; we don't hold long technical planning sessions for how to design a few hundred lines of code. We'd likely hold a meeting on the product behind it, but that's not what we're talking about here.

Does that mean it was a failure of leadership? I'd argue No. I would argue that it might be a failure in mentorship, which is different, but in all the startups I've been in, there's one constant: Engineers have MASSIVE freedom and responsibility to get the product out the door. We don't micromanage the implementations. That doesn't mean they always turn out great, and that's where mentorship comes in; teaching how to fish instead of the seniors doing it for them.

> This feels like a false dichotomy to me. Your system existed in a state somewhere between between 4 routes and hundreds, didn't it? At the point something starts being painful is when it should be addressed, imo. That was probably somewhere in the teens of routes would be my guess. At that point the system is still small and understandable enough to make a large architectural change, but it's big enough that you can be reasonably certain you're making the correct large architectural change.

I think you're right. No one is perfect.

The reality of many startups is that the senior developers are vastly overworked. I mean, everyone is overworked, but at least in my experience, it comes back to the fact that the seniors can't micromanage everyone. So at some point, we were at a dozen routes, and it probably sucked to work in, but its likely that the seniors reviewed it and "missed the forest for the trees". Yeah, that code looks great, ship it, next feature, keep growing.

It takes a different, holistic perspective to see that the whole architecture is slowly getting bad, and often you don't experience that different perspective working in the weeds on each route. Combine that with the fact that startups tend to move REALLY fast, and an engineering department four the size still wouldn't have the capacity to take on all the feature work we want, let alone the refactoring, and its easy to understand why architecture gets pushed off.

This logic is a great justification to commit many unnecessary tragedies, because it gives you an escape valve to not put effort into anything you don't want to do.

"It's more complex to write unit tests/write this so it's easy to unit test, so I won't bother".

"It's more complex to spend a few days thinking through the design of this system, so I won't bother."

"It's more complex to write this in a way that can be subclassed later, so I won't bother."

Perfect may be the enemy of good, but "easy" can also be a foe.

I think that is a misunderstanding between the problem and the solution. The ideal scenario is to take a complex problem and devise a simple solution.

Problems are not made more complex. They are literally the requirements provided. From an economic perspective, the problem is. The complexity is in how a person receives those requirements (perception of approach). This is lost on someone when a problem is more fully exposed according to potential second and third consequences (risks). When everything is ad hoc the risks are never exposed until a mountain of code is written that ends up being not simple and excessively fragile.

To avoid the unforeseen stupidity I prefer to expose the complexity up front and try to prevent people from hiding from it. That doesn't change the problem statement, the environment, the end user, or anything else. It only changes the investment of effort and the person providing that effort. Burying your code in layers of unnecessary abstraction and mountains of dependencies is not exercise in simplicity because somebody has successfully avoided writing original code. This is the difference between simple and easy.

>I deliberately try to make things complex

I see this often with someone who has made it into a senior role and then is attempting to ensure job security through complexity. Reflecting this on to junior developers further reinforces that said developer has adopted (usually subconsciously) a fear-driven approach.

Job security does not seem like a reasonable explanation for programmer behavior in this day and age.

Yes and no. Some developers really want to stay in the same place working on the same sorts of things and to have control over the domains they are working on. There are usually many reasons for this, some ones are easy to understand such as people who value autonomy in their work over the pure dollar value of salary that could be gained by job-hopping. For some folks, the idea of having to go somewhere new with different practices and where they may not be in charge of making decisions on the codebase (or at least not having any control over what goes into it) is terrifying.

I don't need to write bad code to justify my existence when my existence is justified in the face of other developers writing such bad code.

> I deliberately try to make things complex, but not more complex than they actually are

I don't understand. Is it an elaborate way to say that you expose your juniors to the full complexity of the problem, rather than feeding them a "simplistic" view of the problem?

> This is more often due to insecurity than accident or technical ignorance and while their motives are often unintentional (non-cognitive) the actions are often exceedingly deliberate (cognitive decisions).

I don't understand. It looks like they are actually "cognitively" doing their job to me, that is they try to separate the essential complexity from the accidental complexity. They are trying to cut the Gordian knot, I don't see how this is linked to insecurity. Maybe you should simply tell them that someone already did that job and they should not second-guess them?

> Again, its all about risk or performance in the product versus the individual developer's confidence displayed as a convenience.

No, the convenience is to do as Mister Senior Developer said, as a good code monkey, so you can leave early and work on your side project.

Economically speaking, the problem is. It isn't necessarily complex or simple. Whether, or not, you are willing to accept that there is are risks and possibly second or third order consequences to your approach is the true complexity.

Exposing the fullness of the problem isn't a creation of complexity. The same amount of complexity is already there unchanged. Are your juniors willing and capable to account for this to devise a simplistic resolution that actually lowers risk and with the smallest amount of code? Conversely are they imposing unnecessary layers of abstractions to remove themselves from the directness of the problem?

Another way to think about it in "junior" terms is whether its simple or easy. Those aren't the same.

I really like this view.

I see problems as a messy graph. There is a node on the far left that represents the problem and a node on the far right that represents the solution. There are nearly infinite ways to traverse this graph to complete the problem, but there is a minimum number of nodes that must be passed, along some optimal path, in order to actually solve the problem, as defined. No less is possible, unless you change the problem. I think being able to see something close to this minimum complexity, and that it is the problem, is what you’re referring to.

After identifying this, the problem/solution nodes can be reassigned intentionally.

I tend to see this in a different way.

In most of the difficult architectural decisions I have encountered, there is a slider that goes from "complex" on one end to "aspirational" on the other. I see this trade-off much more often than I see things getting over/under-engineered, though I've definitely also seen that.

"Complex" tends to be actually easier to design: you adapt the existing somewhat complex system, add a couple things here and there, and you have a workable solution to the problem at hand, even if it feels a bit kludgy.

Simple tends to be harder to achieve because reality is complex, and designing a small number of abstractions to encompass a complex system requires willingness to spend blood and sweat in "meatspace" to reign in the complicating factors. Don't want to support XYZ? Well, you're going to have to get people to stop using XYZ first.

I don't think this extends beyond software architecture, but that has been my experience working on living codebases.

You all have it wrong. There is a solution to building better systems.

Complexity nor simplicity are the answer for future proofing work. If a problem is complex then the problem requires a complex solution. Simplification can only be done on problems that are inherently simple.

The key for future proofing work is modularity. You must design a system in small components and each component should be self contained and unaware of the other components. Additionally, the way these components are connected together matters. The goal is to try to build a system that is a linked list rather then a graph of complex nodes. Systems that look like linked lists can have nodes removed and inserted easily in the pipeline while graphs cannot. Composability and unidirectional data flow all lead to modularity. Do not build graph architectures, build pipelines whenever possible.

Additionally the nodes in the system must never shift. If a node is a functional unit then passing closures and first class functions all over the system is equivalent to a graph structure that shifts with time. A shifting graph is not modular at all.

This is key. A simple or complex solution cannot adapt to a changing problem; and if modularity is not introduced into the system then hacks or rewrites need to be made to the entire module to get things to work. Keep in mind modularity does not mean generality. You don't need to create abstract general systems to create modular systems.

Of course this model doesn't work on all problems and there are always compromises. The goal is to minimize the compromises. GUIs are problems that doesn't fit the problem well. You will note that Redux attempts to solve this issue by attempting to transform the GUI problem into something as close to a linked list as possible. However a full transformation isn't possible due to the inherently complex graph nature of the GUIs' itself.

The very model of OOP is the antithesis of modularity. While one can create a lot of abstract objects that can be repurposed for other uses this modularity is an illusion. OOP actually promotes a style of programming where your code becomes a graph of nodes leading to less modularity overall.

I have the opposite experience. I think a senior should focus not on identifying the complexity of a problem, but identifying the complexity of the proposed solution. What are the bottlenecks? What possible situations wouldn't fit into the proposed schema? Would a table of 1k records that have a seemingly large number of columns suddenly become 10m records because you decide to use an EAV?

To mitigate issues, is a solution full of contracts and complicated dependencies the right answer? Or is the right answer to not use the ORM for that one particular call and just write a parameterized SQL query or proc that gets the job done?

That kind of insight upfront only comes from experience, which is what a senior has to offer most of all.

> I deliberately try to make things complex

This strikes me as odd. You really sit there thinking "I need to make this complex"? Why not strive for correctness and completeness?

More like I need to explain what the scenario actually is opposed to the microscopic slice the developer wishes it were. I don't makes a problem complex. A problem is a summation of the requirements before you whether or not you believe it complex.