Hacker News Reader

Comment by somehnacct3757 | Hacker News Reader

I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written.

Maybe the company was betting at the time on an integrations marketplace, or a lot of partnerships, so a robust integration platform was built. Then the company moved on to another bet after only writing one or two integrations on the new platform. Nobody over-engineered here. Everything was built to spec. But three years down the line the new dev is going to get assigned a bug in one of the integrations and lament how the code was over-engineered. Hindsight is 20/20.

Lots of stories from the trenches including many in this thread make this mistake. The same goes for 'tech debt'. If you weren't there, you don't know.

View on HN

I think the lesson here is that with great product market fit everything is under engineered, with poor product market fit everything is over engineered. Statistically you are more likely to be building an over engineered product with poor product market fit than you are to be building an under engineered product with great market fit.

Now is over or under engineering a bad thing? that depends on how many resources the company can muster, your customer tolerance to bugs/failures, and whether you can build a small subset of features well. In practice I've observed products with fewer, but well built features tend to succeed more so than buggy featureful products. Large companies may opt to overbuild everything simply so that they can prove that the product was bad rather than having to decide whether it was product or execution that failed.

+1 to less better built features. What I primarily see kill products is going after too many features and they all suck.

Often when you do one thing really well you may find there is more value in the primitives

Or going after too many users. The 'next billion' mindset is misguided.

I politely disagree.

I'm currently walking away from a code-base (and the company who let it be built) that was obviously built by someone who's mental model of how the code works was incorrect. At some very basic levels, it under-performs compared to literally every other example of this sort of code I've ever seen (and it's extremely common), which results in a subpar user-experience and an EC2 instance way bigger than necessary to run this sort of code. A lot of code running doesn't need to run in the first place, and due to the incorrect mental model, there's a substantial amount of code that doesn't run.

Nobody really understands how bad it is, though, because it's barely tested (and the engineer who wrote most of the code has argued that we don't need tests, for various reasons) and all of the engineers on the team who didn't build it initially, have told me that they're not familiar enough with how this sort of product should work to understand that it doesn't work like that (again, this is a super common kind of product).

There's a lot of other anti-patterns (I started keeping a list in my first week) but I think these are the most damning. This code is in production. Nobody at the org brought up these issues earlier in development (like saying "hey, nobody else's product does [annoying thing that doesn't need to happen], why does ours?"), which leads me to believe that the whole technical org is at fault.

It sucks and I can't wait to be done with this experience.

That doesn't sound like over-engineering. That sounds like bad-engineering.

I don't disagree, but I think that the reason that this fits here is because the poorly engineered software is unnecessarily complex (not in the "someday we might need this" way, but in ways that I'm having a hard time articulating without giving away the product because I know my coworkers are on HN). In fact, it was the ridiculous complexity that drove me to discover the disparities between the original developer's mental-model and the working-model. It just so happens that I've worked on this sort of thing before (as have thousands of others) that I was pretty quickly able to understand where the original developer went wrong (trust me, I'm just an ordinary developer, not a 10x elite hacker or anything).

You see, overengineering is exclusively of the "some day we might need this" kind of complexity. Anything else is a different problem, very likely with very different causes, even if the consequences are similar, so it's not useful to analyze together.

There is also some of this.

Some of it makes sense when the reasons were explained (e.g. "we figured this would make it easier for [business unit] to do [thing] which turned out not to be something [business unit] actually has interest in doing").

Other examples are purely of this sort (e.g. "we'll be glad we have this in place when we finally hit [x] users or enter [market]").

Feature wise, when the business asks for it, it is not over engineering, but when the dev team tries to out think the business team then it is.

Design wise, I know it when I see it but other people may see different things.

Interesting angle. What if the business asked for it, the dev team delivered it, but the business subsequently gave up on it, and now it's overengineered in comparison to what remains in use. To avoid this scenario, I don't try to outthink the business team, but I do push people toward temporary manual workflows sometimes, and promise to automate later if the idea actually pans out.

Yes the business team will often ask for things they don't need but I can justify that cost. It can still impede other development but we tackle that when it becomes a problem not before.

I agree completely on pushing off features that are not core because it is better to learn in production than to daydream in meetings. I would say most phase-2 features never get implemented.

Typically over-engineered software looks something like the following.

1) The product is horizontally scalable to 3 orders of magnitude more traffic than the product will ever receive.

2) Bespoke assembler/hand crafted memory management/other do not touch code to shave 5ms off a 5 ms call on an API that isn't latency sensitive.

3) Ability to handle N customer tenants while only handling 1 customer 3 years later.

4) multi-region redundant deployment for 1 customer strictly based out of the US with no availability requirements.

5) 100% coverage for integration tests, unit tests, CI/CD, for a single tenant blog that is never updated and no one reads.

6) Custom internal framework with micro-service splits to support team of 100 engineers working on app when there are 2.

7) Automated continuously trained ML ranking model which performs .001% better than sorting by popularity or other trivial heuristic on a social media app with 10 users.

The common theme in many of these cases is that these are pretty good ideas if you have unlimited people, but have fairly low impact. In some cases these may be counter-productive to successful delivery at the company/product level. A piece of software built wrongly due to poor understanding of the product domain is often considered under-engineering.

+1. An over-engineered software is a bazooka when you need a hand-gun, not a malfunctioning handgun!

Isn't the common theme: "this is what big tech does" ? Ie, it seems a lot of small companies/startups look at "best practices" and just copy them. It's not just engineering. It happens in product, hiring, marketing, everything.

It's completely wrong given the context 99% of companies are operating in. It's no wonder a Zuckerberg can come along and build a crazy successful company in his early 20s. He likely didn't know any "best practices". Common sense was enough.

> Isn't the common theme: "this is what big tech does" ?

In my org, we love to call this out with an (often private) "We need to do it because Google Does it!" -comment.

I feel your pain :)

Complexity kills projects and the will to work on them. It can be caused by either over or under-engineering. With the first you end up with extraneous layers of abstraction and badly applied design patterns that make it hard to understand or reason about the code...in the second you end up with a big ball of mud that's impossible to understand or tease apart.

In both cases the code gets harder and harder to change to the point that no one really knows how it works or wants to risk touching it.

Over engineering is a form of bad engineering, at least in my book. Fun starts when some parts of a product are over engineered while others are under engineered... One of the cases good averages get you nowhere...

I think nearly everyone would agree that "over == bad" in most settings, but the reverse is being implied by the GP - that bad engineering is a form of over engineering.

No tests? Nobody knows how it works? It doesn't perform well and doesn't fit the problem it's solving? That's very much not over engineered in my book.

Sounds like there are multiple employees. Does the product generate the revenue for their paychecks? If so, it's working correctly.

Love/stealing this, but adapting it to be:

> Does the product generate the revenue for their paychecks, plus extra for the company?

You want to be comfortably generating/contributing more revenue than the expense of employing you.

I appreciate this sentiment until I am assigned to maintain and add features to this product.

I understand the sentiment but there are so many situations where this is not true.

All it takes is a good enough sales and marketing team, or the right executive relationships, especially if the team being “supported” is small.

You can say the company or division is presently functioning, but the time constant on the product is different, and the coupling is too loose to say anything else with accuracy.

> All it takes is a good enough sales and marketing team, or the right executive relationships, especially if the team being “supported” is small.

That's the point though. Getting caught up in engineering purity or even whether it's objectively _good_ is a waste. The product exists to drive business (financial) metrics. If it's doing that, it's working.

The one thing that I'd challenge about this line of thought is that the product isn't necessarily "done", there are expectations to add features & fix bugs.

If the product is poorly crafted, and especially if it's also poorly tested, adding features can introduce bugs, and fixing bugs in one place can cause issues elsewhere. Development then slows to handle these occurrences, which can then impact product viability.

I'm aware of technical debt and it's consequences. The point is that the vast majority of developers treat their job as creating good software, when it's absolutely not.

I'd say it's creating software that is "good enough" (good enough to do what it needs to, right now, sufficiently frequently that paying customers return).

You've checked-out, so it's too late, but depending on the circumstances, this can be a situation for an experienced engineer to step in. The biggest challenges are political, but also triaging the issues and coming up with a migration plan. "Rewrite all the things" rarely goes well, especially if they already work, just on beefier hardware than is needed.

Yeah, I have no interest in being a "hero" or whatever. If they want a lead, they can hire a lead (I wasn't hired as a lead).

> I'm currently walking away from a code-base

> and the engineer who wrote most of the code has argued that we don't need tests, for various reasons

Sounds like you made the right choice to walk away. "My code doesn't need tests" smacks of match-over-the-shoulder type of chaos creation. This person has likely never had to maintain their own code or someone else's, nor has anyone ever had to maintain their code. 0/5 would not work with.

Tests are wonderful. Tests are proof your code works. They are something to be proud of. And they defend your code against the chaos monkey of future people refactoring and rearranging.

What product ?

> I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written.

The specs and the decisions surrounding the code are usually part of the over-engineering.

> Everything was built to spec.

Even with reasonable specs, I've worked with, and later managed, engineers who had a strong drive toward over-engineering solutions. I've also done it myself, and I suspect most people reading this have done the same.

The problem is that many of us engineers became developers because we enjoy building things and we enjoy solving complex problems. It can be exciting to imagine a particularly complex version of a problem and how elegant and fun a complex solution would be.

It can be as simple as a programmer who tries to write their own CMS when the spec could be solved with a simple blogging platform. If you're an ambitious programmer, writing a CMS is fun! Installing Wordpress is blech.

I didn't truly understand the ubiquity of the problem until managing ambitious engineers who wanted to do complex things. At times it felt like 1/4 of my job was just asking "If we wanted to simplify this solution, what would that look like?" Repeat for a few rounds and eventually the team has shed a lot of unnecessary complexity without sacrificing the spec at all.

With the benefit of hindsight, I think you can conclude that the system was over-engineered. If you’re trying to answer only “was it over-engineered?” I think you can answer it with hindsight.

If you’re trying to answer “did the people who built it exhibit poor judgment resulting in the system being over-engineered?” then you need to know more than just the after-the-fact outcome.

In my experience, the situation you're describing more often results in "sloppy" code -- it seems convoluted, non-obvious, poorly thought out until you realize when the pivots happened. Overengineered code usually looks neat at first glance, maybe even impressive, especially to non-technical stakeholders. Then when you actually dive into it, you notice a bunch of wrong abstractions that introduce layers of indirection without offering any sort of valuable flexibility. "This metaclass that implements a context manager could've just been a slightly longer than average function" type stuff.

A codebase that pivoted multiple times makes more sense the more time you spend in it, an overengineered one makes less.

It's a question of framing. Nothing is ever over- or under-engineered. It's over- or under-engineered for a purpose.

What that distinction recognizes is that it's possible for something to have been well-engineered at one time, while still being over-engineered today. Header files in C and C++ are an example of this phenomenon. They solved a very real problem with technical constraints from 40, 50 years ago, both in terms of computers' capabilities and compiler technology, but, since those problems don't exist anymore, they function as over-engineering.

Agreed, but "nothing is ever over- or under-engineered" is a bit too bold. There is definitely over engineering just for the sake of over engineering, and under engineering caused by incompetency.

What is the less-engineered alternative to header files?

Symbolic imports.

Basically, the way every programming language newer than C++ or Objective-C does it.

> If you weren't there, you don't know.

Technically speaking, for any given situation there's no way you can be wrong.

But this post describes a distinct and, by my experience, dominant industry phenomenon. And the stakes are as high is OP says (dead products).

Industry engineers in the large are massively divorced from product outcome. This should be no surprise, as it is a seller's market:

Delivering product simply and directly can be, depending on perspective, somewhat of a mundane and Sisyphean activity.

Engineering "stuff" is far more intellectually engaging, and there's no market pressure (for engineers) against over-engineering. If my company fails to deliver a product outcome, whether a viable company or startup, in this market there is no recourse at all on my salary let alone my hireability.

I say "depending on perspective" because if you can find a way to be more intellectually and emotionally stimulated by shipping stable product over time, then ime you _can_ unite your joie de vivre with market outcomes.

But that does not at all describe most engineers in industry who generally at best shed off some market value while spinning unnecessary webs of toy abstractions (ie liabilities).

My new colleague used to wonder about certain things i.e. why is this is done this way, it doesn't make any sense. As I had been there a lot longer, I would share the technical and non-technical background/restrictions we operated with. Eventually when another new colleague joined, he told the guy, "I used to wonder why some parts of the code are setup that way; now that I have the background I can say that if you are wondering about those same things, trust me, there is a reason/background - its not because the people who did it that way were stupid".

Occasionally, it is because of that though. If not 'stupid', at least inexperienced. I've had plenty of things I've done that worked, but were, in hindsight, 'stupid' (and have been called out on that). Sometimes, people try to make a lot of post-hoc justifications for a block of code or a data/tech decision that really is, just... 'stupid'. Again, that's more likely down to inexperience than anything else, but not every decision is a 'good' one just because people 'had their reasons'.

Examples

Having a 'user info' table with 190+ boolean columns for 'country' ('US','CA','DE','IT', etc) in case someone wants to indicate they're 'from' two countries.

Joining views on views of joined views which are themselves built on joins of other views is, likely, not a terribly sound data decision. (X: "It worked fine last year - you must have broken something." Me: "well... last year you had 450 users, and now there are 37000 users - this isn't a performant way of doing this". X: "Can't be true - the person who wrote it is a DBA")

> Can't be true - the person who wrote it is a DBA

Not for long…

heh... well... he was a DBA, and wrote all the application code as well. testing on localhost with 20 sample users, what could possibly go wrong? :)

> Occasionally, it is because of that though. If not 'stupid', at least inexperienced. I've had plenty of things I've done that worked, but were, in hindsight, 'stupid' (and have been called out on that). Sometimes, people try to make a lot of post-hoc justifications for a block of code or a data/tech decision that really is, just... 'stupid'. Again, that's more likely down to inexperience than anything else, but not every decision is a 'good' one just because people 'had their reasons'.

I agree on the point regarding the decision being 'stupid' but in this case it was not because of inexperience. On the contrary, it was because an overruling decision by an experienced manager. So the point is that it is not always technical or due to inexperience (though that does happen) - from what I've seen, it is quite common for such things to happen due to hierarchical/ego/political issues as well.

you're not wrong, but there's a whole class of problems that occur outside 'enterprise' structures. Often it's just a lone cowboy building something for a small business, and the business owner have absolutely no way to determine if what's being delivered is 'good' in any meaningful sense.

The ever-relevant Chesterton’s Fence.

https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

Can you prove it in a court of law? No. But you still can often easily diagnose it. And you'll usually be correct.

For example the conditions under which https://wiki.c2.com/?SecondSystemEffect happens are well-known, and the pattern has been repeated often in the half-century since it was named. If it is a second system, and you can find complicating abstractions in it that get used exactly once in practice, then with better than 90% odds it was over-engineered. And the farther you dig, the stronger the evidence will become.

Specs are often over engineered. This is part of the problem. Too much effort in rowing the boat, not enough effort in figuring out the right direction to point it first.

This is why commit messages are so important. Documentation that is external to the code its referencing will always drift but a commit message is a snapshot in time that is attached to the thing it is referring to. In my experience not enough people take commit messages seriously and just basically give a summary of what changed not why it was changed.

Commit messages get lost too easily. It’s the worse way to document reasoning around the code. There’s many proposals to document code outside of code and none of them has ever worked. Just comment your code.

> Maybe the company was betting at the time on an integrations marketplace, or a lot of partnerships, so a robust integration platform was built. Then the company moved on to another bet after only writing one or two integrations on the new platform. Nobody over-engineered here. Everything was built to spec.

I would argue this is a case of over-engineering. Though, it extends beyond pure engineering.

This is poor resource management and business de-risking. An effective approach here would have been: "Let's build out a handful of integrations. Make sure they're going to deliver the value we want. Then abstract it into a repeatable pattern".

I've been in exactly this position, in fact I conceived and created the situation at my last company. I found it uncanny how on point OP was.

The trouble with "Let's build out a handful of integrations. Make sure they're going to deliver the value we want. Then abstract it into a repeatable pattern" is that when you're building an integration marketplace, you need partners, and they don't want to be your research bitches.

They just want to build their integrations once, and gtfo. If they think you're signing then up for a revolving wheel of experimentation, they'll just politely stay away until someone else has done the hard yards.

Sure if you're a Microsoft or a Google you'll have any number of willing partners who will put up with anything to be the pioneers in your integrations marketplace.

But otherwise, they're using your integrations marketplace purely for the benefits, and they don't want to be building on sand.

> Everything was built to spec.

First line in the second paragraph of every dead company's obituary.

Well how do you know something is overengineered? It's when you know it will never (or too far in the future) need to scale accordingly to the engineering.

Reply on news.ycombinator.com