Hacker News Reader

I had a phone interview the other day - it was set up through hired, an online platform where recruiters ask you to interview. The recruiter used “good time to meet” to set up the interview which I then had to accept in a Google Calendar. When he called me my iPhone helpfully automatically sent an unknown caller silently to voicemail but I saw it ringing and tried to pick up, but too late. I then called the number back - it was a Google Meet computer voice that asked me for a PIN number rather than being the recruiters actual phone. I didn’t get any PIN number and the calendar invite didn’t have one. The recruiter didn’t provide his actual phone number only an email. I tried sending an email but didn’t get a reply. Knowing that he was using Google Meet I tried to call him by Google Meet but there was no answer. I wasn’t too bothered by it but I marveled at all the stupid layers of unnecessary technological complexity stacked on top of each other that had led to this situation. Is all of that really necessary to make a phone call?

I have been recently hiring and the easiest process for me is just to receive an email to our hello@dyvenia.com address. I can start building rapport and communicate easily with potential candidates. All I would need additionally is an automated assistant to store the cvs and my feedback and send reminders, it could all be done with email.

I need to go to the bathroom. The simplest thing that solves my immediate problem is to urinate in my pants. I ate a bag of chips and now I have an empty bag to dispose of. The simplest thing that solves my immediate problem is to throw it on the floor.

So it's clear that "the simplest thing that solves my immediate problem", like simply adding a new int field to the most convenient table, can compound into an awful mess. But perhaps "simple" is not the right word here.

I like Rich Hickey's talk on simple vs. easy; we're both using the wrong word according to him. "Simple" means not intertwined or tangled; well-organized. "Easy" means "close at hand" or "familiar". We both mean "easy" here.

https://www.infoq.com/presentations/Simple-Made-Easy/

That being said, your examples of complexity fetish do indeed sound awful. Abstract classes, optional configuration files, environment variables and regular expression; we can agree those are awful. Those are neither easy nor simple. But the problem is that they're not discussions about the domain, they're truly unnecessary. Maybe that's all you really mean.

>We had to add something to the database the other day. Big argument. Should be one to many? many to many? what if this or that happens? what if requirements change? You know what - for the requirement we actually had it was solvable with a single integer field on an existing table.

Agreed about not inventing requirements, but questions about "how is this likely to change in the future?" are much closer to productive discussion. Discussions about one-to-many vs. many-to-many can also be the exact discussions software developers should be spending most of our time on (although don't get me started on the awful database designs most software has, so these discussions may be inane for that reason alone).

> I ate a bag of chips and now I have an empty bag to dispose of. The simplest thing that solves my immediate problem is to throw it on the floor.

Maybe that's the best solution for the long run, instead of designing and implementing a whole garbage disposal system from the ground up for only one piece of trash.

My problem a lot of software developers are trying to solve problems they don't have and never will. This consumes time and adds unnecessary complexity to their projects.

People are bad a predicting future. Especially when this predicting is done in 5 minutes before implementing something and not dedicated activity which includes interviewing of users and domain experts.

I've seen this many times: programmer is asked to solve a small and well defined problem. Instead programmer generalizes it and makes something more universal with the requested feature as a special case. More often than not nothing except this special case is used.

Or working on some new project they add a feature which looks useful in theory, but ends up being rarely/never used. It may look easy to implement initially, but over the years maintenance cost can be much higher.

People are amazing at predicting the future, and in some ways we are better at it than remembering the past. That's because we use the same machinery to do both. We partly remember the future, and predict the past. This ability breaks down with complexity and abstractness, as well as with novelty, all of which are involved in software (I can tell you that the sun will come up tomorrow, and where I should move my hand to catch a ball, but I can't predict all of the defects my software will have--though if it involves X.509 certificates, I can tell you exactly when a particular sort of outage will occur)

Prematurely designing for scale when I just needed to finish the beta version has been my engineering vice.

For me too this took years and years to learn. It's a hard lesson that seems can only be learned by walking the road and learning from working on a particular piece of software for a long time, at least that's what it took for me.

I guess it's called experience to know when to design and when to just implement. Somebody wrote somewhere for example that if you're not going to need a particular piece of code in more than 3 different places, don't write a function for it.

As a newbie you would totally want to write a function for it, thus also making it harder to read the code as you would have to understand the function in order to see what it does in that context.

Also thinking in terms of "Do I really need this feature in future use cases?" is something I don't feel you can assess when not having the experience of already have peeked into those future use cases, where in many cases you will not ever need that particular function in more than this one place.

But can you learn how to design a reusable system without first doing it in the wrong places ? That's something that is hard to say, I don't know.

Could you teach somebody who wants to build complex, reusable components not to do it and just stick to simplicity ? How would one then know how to build those reusable systems where you need them ?

Maybe we should focus more on training both simplicity and complex design, but rarely you can do that when you are under pressure and working on real life software.

Haha, on the other hands a lot of developers never grow out of always developing betas and their only concept of programming is developing betas and then dealing with fires all the time.

aka YAGNI

Haha, touché; I thought I had come up with completely unassailable examples of obviously bad choices but you've made a good point that a single piece of trash on the floor may occasionally be the best option. Engineering is all about tradeoffs, even in the extreme.

This is why my dream is to run a one man software business at some point. At least then when things have been stupidly over engineered I'll have only myself to blame.

I don't have many years of writing software on a team left in me. It really manages to suck all of the joy out of both the process and the result.

You have to remember that for employees or perhaps aspiring employees, solving the problem is rarely the only priority, if it is one at all.

Impressing others on the team, impressing the boss, making complex stuff for the resume, seeming like a contributor in meetings, having interesting tasks, etc.

>having interesting tasks

I've definitely over-engineered things at work because I was bored. There's no reason the login form for an internal tool with six users needs to handle 10,000 requests per second

What extra did you do to allow it handle 10k requests per second?

>Impressing others on the team, impressing the boss, making complex stuff for the resume, seeming like a contributor in meetings, having interesting tasks, etc.

As a boss, I feel like I'm VERY close to a life lesson here.

Put most simply, there is very often nothing that connects an engineer’s best interests to how much the customer likes the product and plenty of reasons that a product might seen as a means to an end by the engineer, most significantly the fact that they probably won’t be there in 1-2 years.

This is important to watch for. You can seriously cramp someone's style if you actually solve problems - especially if it's something being milked for complexity theater. Solving problems strategically can be very impressive to management and destructive to gold bricking colleagues.

I think that once you reach a certain point, you stop believing in magic solutions to all your problems.

You’d think you find this out over the course of a single project, when every new thing you introduce requires extra pain to support in the future, but in reality it takes years of this happening many times before you internalize that.

I do think it’s a bit dependent on the organization you work for though. If you know them well, you’ll realize that adding the single integer field is not going to cut it because they’ll come back in a month asking for the 1 to many relation (if you try to clarify in advance, of course they tell you they don’t need it).

I fully agree with not building for requirements you do not immediately (next month or so) anticipate though.

That said, as you say, I’ll probably go off and add some unnecessary complexity somewhere when I go back to work tomorrow ;)

"The epitome of genius in my mind is finding the solution that solves the problem without introducing anything at all."

Agreed. Albert Einstein once said "Everything should be made as simple as possible, but no simpler." This tautology sounds a bit silly, but I think the point is that the escalation of complexity needs to be fought with intentional focus.

That said, I think abstraction can be a useful tool for fighting complexity, even if introduced before business needs demand. In other words, developing a series of independent one-off solutions could add more complexity to your system in the long run than finding the right conceptual model that unifies them proactively.

> proactively

And here's the problem. 90% of the time pro-active abstraction (complexity) is never needed and therefore bad.

You can always do things retro-actively.

If you find yourself writing the same thing 3 times then yes, by all means abstract it away.

Good abstractions are not invented; they reveal yourself to you in due time.

> You can always do things retro-actively.

Doesn't this vary depending on who consumes your code and how easily they can adapt to change?

Good abstractions don't just exist to reduce code reuse; they help people reason about software. They turn logical operations into higher order concepts that are easy to think about. If this weren't true we would write all our programs as single, very long functions.

The right abstractions will reduce complexity not add to it. Consider the following information architecture:

Cats are warm-blooded animals that produce milk and give live birth. Dogs are warm-blooded animals that produce milk and give live birth. Humans are warm-blooded animals that produce milk and give live birth.

vs

Mammals are warm-blooded animals that produce milk and give live birth. Cats are mammals. Dogs are mammals. Humans are mammals.

I would argue the latter is less complex even though it contains an abstraction.

Also, you can't always fix things retroactively. As an IC, you don't control what efforts are funded. Not all organizations invest significant resources in cleaning up tech debt, even it helps long-term productivity.

This works until the analogue of monotremes shows up to ruin your supposedly flawless categorisation.

YAGNI

I didn't say the categorization was flawless; I said it reduced the complexity of the information. Categories can be useful without being flawless.

It is not a “can be”, it’s a “must be”. Models or an abstraction of complexity are necessarily both reductionist and useful.

> The right abstractions will reduce complexity not add to it.

Platypus joined the chat.

Monotremes are mammals that lay eggs. Platypi are monotremes.

(If you like, you can change the definition of “mammal” to say “give birth to live young, except for monotremes”. That's exhaustive.)

I think the point of the thread is that you're probably not going to need a platypus or a human or even a dog. If the requirements are that you support a cat only, then you just need a Cat class and no big abstraction taxonomy.

>Good abstractions are not invented; they reveal yourself to you in due time.

Love that quote, that’s usually how it goes for me.

This sounds like Einstein the mythical quote-originator, rather than the human scientist. https://quoteinvestigator.com/2011/05/13/einstein-simple/

All the same it's a good aphorism.

See also:

> "Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away"

- Antoine De Saint-Exupéry

Also:

C.A.R. Hoare:

“There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.”

Apparently this is what Einstein has been documented saying:

"...the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience."

So I guess the supposed quote is paraphrasing.

It's certainly got the same vibe. I certainly don't want to begrudge Einstein credit here. If you have the dubious honour of knowing physics and German, then reading his work (e.g. [0]) will leave you in no doubt that he was a shining exemplar of this principle.

He does get a ton of stuff misattributed to him though.

[0] https://onlinelibrary.wiley.com/doi/10.1002/andp.19053220806

Simplicity and complexity are terms we use a lot to argue that something is good or bad, but nobody agrees on what is or not simple or complex and how that should be evaluated.

As an example, both Clojure and Go are programming languages that present themselves as focused on simplicity, yet they are in complete disagreement with each other. They have a completely different understanding of what is complexity and how to approach simplicity.

Because of this, when I see someone saying that something is "simple" or "complex" I personally consider that they really express how familiar they are with the approach or concepts used. While we can all agree that modern software is clearly overly complex due to the massive number of moving parts and opportunity for bugs and other issues, I don't feel that we have a good framework to actually evaluate that complexity in a way that could then be discussed in a more serious, analytical way.

Yep, the danger of becoming a zealot and simplifying components too much is it can increase complexity of the system as a whole. It's not an additive property, you need to look at it from different angles and layers and understand how it leverages. Gos lack of generics being a common example of this. Another example: UDP is less complex than TCP - until you experience packet loss and reinvent re-transmission yourself in the application layer.

"I know everyone thinks they like simplicity but my observation is that about 95% of software developers have a complexity fetish."

You do not have to be working in the software industry to see this, either.

It's no wonder the majority of these complexity fetishists are paid either from advertising (like journalists) or from venture capital. There is not much careful thinking behind those sources of money. Its "dumb money". Further, no one in their right mind pays to license the "solutions" these "devs" are producing. That is why almost all of it is free. The stuff that is licensed for a fee is marketed to birds of a feather, complexity fetishists within organisations with the power to dictate purchasing decisions.

If the dumb money dries up, the market for the complexity fetish may suffer. Common sense may prevail. Complexity fetishists may find no one is willing to join them in their delusion of "simplicity". If the dumb money keeps flowing then disregard this comment as irrelevant and carry on with the delusion. The parent comment seems to indicate not everyone is fooled, though.

Completely agree. People almost always underestimate the true cost of adding new frameworks/libraries/configuration files to a project. And they almost never remove things that turned out to be unnecessary. And then they don’t understand why it is so hard for them to maintain the software, looking around for something else they can add to “solve the problem”. Face palm moments galore.

This not unique to software though. We see the same with the law. Lawmakers often underestimate of even ignore costs imposed by a new law. And old (often known to be bad) laws are repealed slower than new laws are created.

May be it is basic human psychology (loss aversion?). People are happy gain something (even not very useful) and unhappy to part with it.

That would make it a framing problem: A loss of complexity can also be seen as a gain in simplicity, freeing up time and resources to be spent elsewhere.

And about laws: I totally agree that it is a huge problem, maybe even a bigger one. Because I have never heard lawmakers talking about technical debt or legacy code. So it seems they are not even aware :)

The personal evolution is very interesting: from complexity zealots to simplicity zealots.

I suggest this has to do with the fact that at very local scale, complexity is not a problem, and individuals are therefore concerned with more of an intellectually elegant solution, irrespective of complexity.

In addition, a complicated but elegant solution speaks to our intellectual passions.

Finally, I think that we compete on those terms far more than we like to admit. Everyone wants to be the 'genius' that others envy, with all the followers, the insightful blogs etc., at least on some level.

As we grow, mature and perspectives and horizons broaden, have little to prove to others, the novelty of a solution may take second set to 'getting it done' and this is when we start to develop a real instinct for the cost of code and complexity. Then we realize that most of our abstractions are leaky, that tons of code is unnecessary and make work, the 'horror' of complexity starts to come into clarity ... and we kind of turn into zealots.

I'm in the simplicity zealot phase and I wonder what's next.

I suggest it's probably just the 'Sausage Phase' wherein we accept what's there, and accept the limitations of individuals and operating environments.

I wonder how we can teach new devs to move through these phases quickly.

I'm in the simplicity zealot phase and I wonder what's next.

I can kind of sense what the next phase is. I don’t know if it will be the case for sure, but being in this frustrated phase inevitably leads to a quietness inside. When you start out the quietness is not there, so you are always in this anxious state implementing everything under the sun. But now in the aftermath of all the internal directionless energy, the somberness makes me want to make solutions. Build that simpler framework, write that simpler language, make that simple tool/product. Not because you feel the pressure, but because you are finally quiet inside to think peacefully.

It’s odd, we always complain about meetings being a distraction, yet, it’s the endless pursuit to live up to the ideal of ‘being a great developer that’s worth the money’ that has always been the most noise. The turmoil manifests in complicated codebases.

My armchair psychology for today.

> , yet, it’s the endless pursuit to live up to the ideal of ‘being a great developer that’s worth the money’ that has always been the most noise. The turmoil manifests in complicated codebases.

I'm going to print this. My preachings as a 28 year old have gone and fallen on deaf ears. But I will preach it till I drop dead - Complexity results from people justifying their paychecks. No more. No less.

That's really smart, but I think the 'directionless energy' is phase 1 i.e. not aware of complexity.

Phase 2, 'anti complexity' is more one of frustration, trying to stop arbitrary things.

I wonder if the 'quiet' is just rising above the noise and accepting it is what is is, and accepting some devs will build to much like Wild Stallions but that they are useful for some things obviously.

>Again, I know 80% of the people reading my comments right here will nod their heads and identify with it and yet still go away and see themselves as exceptions to the rule

We need people like you. If I could, I would hire you and make you "unfireable". Your job would be to go around and make people do the ABSOLUTELY most simple work.

You could start by shadowing me and yelling at me every time I attend a meeting that I don't need to be in.

i don't think anyone is ever consistently correct about which pieces of complexity are the right ones to take on to make things simple. worshiping & empowering someone who'se only message is that other people are inferior & crude & bad, & using that to assume they must be the right proper judge- the platinum citizen- is in my view an abominably horrific way to enforce a narrow viewset.

and it's not who i nor those engineers i respect want to work with: the fear minded. the Muse denialists, those whose bit is to tell you to reel it in.

I'm ancient by tech standards (lol), but one of the things that keeps me in analytics/data science/stats is that I've always wanted to focus on the problem, not the technology. tech is a means to an end, ideally.

But even in my field, culturally we're often under constant bombardment by people internally and externally with new abstractions, tech, brands, techniques, buzzwords, frameworks, methods, committees, collaborations.

The war is literally never-ending...

If it was JUST human nature or something, I think I could silently let it slide, but imo there's a level of maturity/seniority where you realise this attitude and phenomenon limits the kinds of problems you can actually solve.

sure, politics and self interest is everywhere, but as the article is hinting, eventually you reach a point where the complexity means your system will fold and be swept away. And genuinely complex problems or thoughts cannot be had or tackled if the foundations of your intellectual or tech tower aren't simple, strong or sturdy enough.

A lot of time would be saved and promotions earned if we just did the dumb simple thing up front. Later on when everything is on fire someone can migrate to the robust complicated version and get a nice performance review out of it, win-win.

I believe you're identifying simple and fundamental, which is risky, because it's not explicit for whom the code is supposed to be simple (reader or writer).

Writing good code for a reader is about abstracting to the fundamental level of atomic logic. But this requires quite a lot of practice and experience, not to mention discipline.

If the code is _simple_ from the writer's point of view it's probably filled with tacit assumptions (future breakage) and lazy decision made to get where this thing just works.

Good code requires insight and planning. _My code_ is written from the top of my head to solve a specific issue. I'm an amateur in programming with a long enough academic education to know in detail how I suck :)

If asked I'd probably say I write simple code. But I don't necessarily intend well structured and stable, just that it was straightforward for me at the time of writing it (+ n days).

I agree. But sometimes the obvious and simple solution is not the simplest one. Taking your example about the database: if your table is over 70 million rows and you are using Mysql < 5.6, trust me: it’s simpler to add an additional table than to run a migration on such a big table that will lock it on production.

A long, but thoroughly worthwhile read on this topic is Out of the Tar Pit [1].

[1]: http://curtclifton.net/papers/MoseleyMarks06a.pdf

It is an unsolvable problem. Arguably the reason pre-NT Windows and pre-X MacOS were inadequate is that they were far too simple. Rewriting the systems and basing them on server-like architectures was absolutely necessary to achieve some level of robustness.

In my experience no amount of zealotness will produce better software. Abstractions are good for things you really need but you never know how the product will evolve and what you really need...

Looking from the other side there are many people here who would argue that NT/2000 was a high point and it has been downhill since.

> ...my observation is that about 95% of software developers have a complexity fetish.

Agreed.

Further, I've been ridiculed throughout my career for my efforts to simplify.

Nominally smart people just can't grasp that simplifying is harder. They'll dismiss the result, not appreciating the process.

No true Scotsman would create another abstract class.

this view leaves the practical & steps into the curmudgeonly & nothing good ever gets done there in my experience