Hacker News Reader

Comment by jcgrillo | Hacker News Reader

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There's this bizarre misconception popular among bigtech managers that there's some tunable tradeoff between quality and development speed. But it doesn't actually work that way at all. I can't even count anymore how many times I've had to explain how taking this or that locally optimal shortcut will make it take longer overall to complete the project.

In other words, it's a skill issue. LLMs can only make this worse. Hiring unskilled programmers and giving them a machine for generating garbage isn't the way. Instead, train them, and reject low quality work.

View on HN

> What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it.

I don't think finding such programmers is really difficult. What is difficult is finding such people if you expect them to be docile to incompetent managers and other incompetent people involved in the project who, for example, got their position not by merit and competence, but by playing political games.

"What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti."

In my opinion the reason we get enterprise spaghetti is largely due to requirement issues and scope creep. It's nearly impossible to create a streamlined system without knowing what it should look like. And once the system gets to a certain size, it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary. Plus the full requirements are usually poorly documented and long forgotten by that time.

When scopes creep and requirements change, simply refactor. Where is it written in The Law that you have to accrue technical debt? EDIT: I'm gonna double down on this one. The fact that your organization thinks they can demand of you that you can magically weathervane your codebase to their changeable whims is evidence that you have failed to realistically communicate to them what is actually possible to do well. The fact that they think it's a move you can make to creep the scope, or change the requirements, is the problem. Every time that happens it should be studied within the organization as a major, costly failure--like an outage or similar.

> it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary

That's a choice. There are some other options:

- Simply don't get business buy-in. Do without. Form a terrorist cell within your organization. You'll likely outpace them. Or you'll get fired, which means you'll get severance, unemployment, a vacation, and the opportunity to apply to a job at a better company.

- Fight viciously for engineering independence. You business people can do the businessing, but us engineers are going to do the engineering. We'll tell you how we'll do it, not the other way.

- Build companies around a culture of doing good, consistent work instead of taking expedient shortcuts. They're rare, but they exist!

> Fight viciously for engineering independence.

Or simply find a position in an industry or department where you commonly have more independence. In my opinion this fight is not worth it - look for another position instead is typically easier.

>When scopes creep and requirements change, simply refactor.

Congratulations, you just refactored out a use case which was documented in a knowledge base which has been replaced by 3 newer ones since then, happens once every 18 months and makes the company go bankrupt if it isn't carried out promptly.

The type of junior devs who think that making code tidy is fixing the application are the type of dev who you don't let near the heart of the code base, and incidentally the type who are best replaced with code gen AI.

Refactoring is improving the design of existing code. It shouldn't change behavior.

And regardless, the way you prevent loss of important functionality isn't by hoping people read docs that no longer exist. It's by writing coarse-grained tests that makes sure the software does the important things. If a programmer wants to change something that breaks a test like that, they go ask a product manager (or whatever you call yours) if that feature still matters.

And if nobody can say whether a feature still matters, the organization doesn't have a software problem, it has a serious management problem. Not all the coding techniques in the world can fix that.

If you don't understand your systems well enough to comfortably refactor them, you're losing the war. I probably should have put "simply" in scare quotes, it isn't simple--and that's the point. Responding to unreasonable demands, like completely changing course at the 11th hour, shouldn't come at a low price.

It's a market for lemons.

Without redoing their work or finding a way to have deep trust (which is possible, but uncommon at a bigcorp) it's hard enough to tell who is earnest and who is faking it (or buying their own baloney) when it comes to propositions like "investing in this piece of tech debt will pay off big time"

As a result, if managers tend to believe such plans, bad ideas drive out good and you end up investing in a tech debt proposal that just wastes time. Burned managers therefore cope by undervaluing any such proposals and preferring the crappy car that at least you know is crappy over the car that allegedly has a brand new 0 mile motor on it but you have no way of distinguishing from a car with a rolled back odometer. They take the locally optimal path because it's the best they can do.

It's taken me 15 years of working in the field and thinking about this to figure it out.

The only way out is an organization where everyone is trusted and competent and is worthy of trust, which again, hard to do at most random bigcorps.

This is my current theory anyway. It's sad, but I think it kind of makes sense.

Soviet vs NATO. The Soviet management style is micromanaging exactly how to do everything from the rear. The NATO style is delegating to the front line ranks.

Being good at the NATO style of management means focusing on the big picture--what, when, why--and leaving how to the people actually doing it.

Agreed.

The way I explain this to managers is that software development is unlike most work. If I'm making widgets and I fuck up, that widget goes out the door never to be seen again. But in software, today's outputs are tomorrow's raw materials. You can trade quality for speed in the very short term at the cost of future productivity, so you're really trading speed for speed.

I should add, though, that one can do the rigorous thinking before or after the doing, and ideally one should do both. That was the key insight behind Martin Fowler's "Refactoring: Improving the Design of Existing Code". Think up front if you can, but the best designs are based on the most information, and there's a lot of information that is not available until later in a project. So you'll want to think as information comes in and adjust designs as you go.

That's something an LLM absolutely can't do, because it doesn't have access to that flow of information and it can't think about where the system should be going.

> the best designs are based on the most information, and there's a lot of information that is not available until later in a project

This is an important point. I don't remember where I read it, but someone said something similar about taking a loss on your first few customers as an early stage startup--basically, the idea is you're buying information about how well or poorly your product meets a need.

Where it goes wrong is if you choose not to act on that information.

For sure. Or, worse, choose to run a company in such a way that anybody making choices is insulated from that information.

It's relatively easy to find a programmer(s) who can realize enterprise project X, it's hard to find a programmer(s) who cares about X. Throwing an increased requirement like speed at it makes this worse because it usually ends up burning out both ends of the equation.

> The problem is finding people to do it with who care enough to think rigorously

> ...

> train them, and reject low quality work.

I agree very strongly with both of these points.

But I've observed a truth about each of them over the last decade-plus of building software.

1) very few people approach the field of software engineering with anything remotely resembling rigor, and

2) there is often little incentive to train juniors and reject subpar output (move fast and break things, etc.)

I don't know where this takes us as an industry? But I feel your comment on a deep level.

> 1) very few people approach the field of software engineering with anything remotely resembling rigor

This is a huge problem. I don't know where it comes from, I think maybe sort of learned helplessness? Like, if systems are so complex that you don't believe a single person can understand it then why bother trying anyway? I think it's possible to inspire people to not accept not understanding. That motivation to figure out what's actually happening and how things actually work is the carrot. The stick is thorough, critical (but kind and fair) code--and, crucially, design--review, and demanding things be re-done when they're not up to par. I've been extremely lucky in my career to have had senior engineers apply both of these tools excellently in my general direction.

> 2) there is often little incentive to train juniors and reject subpar output (move fast and break things, etc.)

One problem is our current (well, for years now) corporate culture is this kind of gig-adjacent-economy where you're only expected to stick around for a few years at most and therefore in order to be worth your comp package you need to be productive on your first day. Companies even advertise this as a good thing "you'll push code to prod on your first day!" It reminds me of those scammy books from when I was a kid in the late 90s "Learn C In 10 Days!".

> This is a huge problem. I don't know where it comes from

I think it's a bunch of things, but one legitimate issue is that software is stupidly complex these days. I had the advantage of starting when computers were pretty simple and have had a chance to grow along with it. (And my dad started when you could still lift up the hood and look at each bit. [1])

When I'm working with junior engineers I have a hard time even summing up how many layers lie beneath what they're working on. And so much of what they have to know is historically contingent. Just the other day I had to explain what LF and CR mean and how it relates to physical machinery that they probably won't see outside of a museum: https://sfba.social/@williampietri/113387049693365012

So I get how junior engineers struggle to develop a belief that the can sort it all out. Especially when so many people end up working on garbage code, where little sense is to be had. It's no wonder so many turn to cargo culting and other superstitious rituals.

[1] https://en.wikipedia.org/wiki/Magnetic-core_memory

I agree as well. These are actually things that bother me a lot about the industry. I’d love to write software that should run problem-free in 2035, but the reality is almost no one cares.

I’ve had the good fortune of getting to write some firmware that will likely work well for a long time to come, but I find most things being written on computers are written with (or very close to) the minimum care possible in order to get the product out. Clean up is intended but rarely occurs.

I think we’d see real benefits from doing a better job, but like many things, we fail to invest early and crave immediate gratification.

> very few people approach the field of software engineering with anything remotely resembling rigor, and

I have this one opinion which I would not say at work:

In software development it's easy to feel smart because what you made "works" and you can show "effects".

- Does it wrap every failable condition in `except Exception`? Uhh, but look, it works.

- Does it define a class hierarchy for what should be a dictionary lookup? It works great tho!

- Does it create a cyclic graph of objects calling each other's methods to create more objects holding references to the objects that created them? And for what, to produce a flat dictionary of data at end of the day? But see, it works.

this is getting boring, maybe just skip past the list

- Does it stuff what should be local variables and parameters in self, creating a big stateful blob of an object where every attribute is optional and methods need to be called in the right order, otherwise you get an exception? Yes, but it works.

- Does it embed a browser engine? But it works!

The programmer, positively affirmed, continues spewing out crap, while the senior keep fighting fires to keep things running, while insulating the programmer from the taste of their own medicine.

But more generally, it's hard to expect people to learn how to solve problems simply if they're given gigantic OO languages with all the features and no apparent cost to any of them. People learn how to write classes and then never learn get good at writing code with a clear data flow.

Even very bright people can get fall for this trap because engineering isn't just about being smart but about using intelligence and experience to solve a problem while minmaxing correctly chosen properties. Those properties should generally be: dev time, complexity (state/flow), correctness, test coverage, ease of change, performance (anything else?). Anyway, "Affirming one's opinions about how things should be done" isn't one of them.

The whole one about the stateful blob of an object with all optional attributes got me real good. Been fighting that for years. But the dev that writes this produces code faster than me and understands parts of the system no one else does and doesn't speak great English, so it continues. And the company is still afloat. So who's right in the end? And does it matter?

I don't know who's right but I know that it's the ergonomics of programming languages that make producing stateful blobs fast and easy that are in the wrong.

You know it's a problem when you have to read a book having couple hundred pages to learn how to hold it right ;)

<< Instead, train them, and reject low quality work.

Ahh, well, in order to save money, training is done via an online class with multiple choice questions, or, if your company is like mine and really committed to making sure that you know they take your training seriously, they put portions of a generic book on 'tech Z' in pdf spread spread over a drm ridden web pages.

As for code, that is reviewed, commented and rejected by llms as well. It is used to be turtles. Now it truly is llms all the way down.

That said, in a sane world, this is what should be happening for a company that actually wants to get good results over time .

> The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it.

There is no incentive to do it. I worked that way, focused on quality and testing and none of my changes blew up in production. My manager opined that this approach is too slow and that it was ok to have minor breakages as long as they are fixed soon. When things break though, it's blame game all around. Loads of hypocrisy.

"Slow is smooth and smooth is fast"

It's true every single time.

Reply on news.ycombinator.com