Hacker News Reader

Comment by nologic01 | Hacker News Reader

Leaner, cleaner, less buggy, more secure, more performant, longer-lived code is obviously entirely possible. If people managed to do it at the dawn of the information age surely they can do it today, with multiple decades of massive experience, not to mention the incredibly powerful tools developed in the meantime.

If its not done its because there is no money in it. In fact the opposite.

The counter-incentives to wasting time on high quality software are numerous and affect all sorts of teams. VC funded startups must get to market first or die. Fake-it-till-you-make-it is their religion. For more mature organizations too, cost and bloat is not an issue. Its a feature. The bigger the team more prestige for the managers etc. The costs are passed on to clients anyway.

How come "ruthless market forces" don't rectify this wastefulness? You'd think that codebases of superior quality will earn the keys to the kingdom. They might, eventually. In a competitive environment that is less prone to pathologies, hypes etc.

View on HN

Kagi is not VC funded, yet we have code that is sub-optimal, a ton of bugs and odd performance issues here and there. Knowing what I know about software development, I do not think this is avoidable, regardless of the source of funding or size of the company. It is a factor of complexity of the software, resources available and incentives in place.

What we can do though, that perhaps VC funded companies can not as easilly, is alocate time to refactor code and deal with tech debt. In fact that is what we are currently doing and we basically pulled a handbreak on all new feature developement for 45 days to deal with our main tech issues. Ability to do this requires a long term thinking horizon. Very difficult to make that kind of investment if you expect to get acquired next year and tech debt becomes somebody else's problem.

Also worth noting, as long as the product is being actively developed it will aways have new bugs and issues. 'Perfect code' is achieveable only in a closed context scenario, where new features are not added any more. (which randomly bring this weird thought to my head, that the only human that does not do any mistakes any more is a dead one; perfection in human actions is only achieved in the absence of life... ok need to stop there)

> which randomly bring this weird thought to my head, that the only human that does not do any mistakes any more is a dead one; perfection in human actions is only achieved in the absence of life... ok need to stop there

Love a good philosophical tangent! Wish you expanded :)

Let’s hope this doesn’t get taken up by a sentient AI in the future :-)

> If its not done its because there is no money in it. In fact the opposite

This bears repeating. It's the disease that has consumed software and is making all modern software the worst possible version of itself.

I think it's the VC funding model which has driven the industry this way. Startups get millions in funding, then it's a race to make enough money to pay back those investors which leads to this. The companies have to squeeze dollars from their app as fast as possible which means anything that doesn't have a ROI metric attached to it will not get a second of anyone's attention.

Honestly, there are many reasons for our current situation. One is that companies aren't (usually) run by engineers; they're run by product or business people. Those types don't care about performance, website footprints, smooth scrolling, etc. They care about adding new features, getting users, and doing so as fast as possible. Another reason is that many web developers were taught that software engineering is mashing together a mixture of Node, Ionic, Bootstrap, Vue.js, Angular, jQuery, etc. to quickly make a website. No one was taught how to do things on their own so they just bundle framework after framework into their projects just to do simple things. Finally, it's not like people built highly performant software in the 90s because they genuinely embodied this article's spirit; they did so out of necessity. As soon as computers got fast enough, we stopped having to focus on micro-optimizations just to get our products to run.

> Those types don't care about performance, website footprints, smooth scrolling, etc

they don't care because their _users_ don't care.

I find these discussions are always led by engineers, shaking their fists at clouds. nobody cares! it doesn't make any money so you're just whining into the void.

That's like saying only engineers care about appliances that work long past the warranty expiration; users don't so business people value engineer it and everyone except the engineers are happy.

Except that's not true. Users do care they just don't have a choice in the matter. I have listened to many, many laypeople who have expressed frustration with software. They may not be able to articulate it to the extent that the quote does, but when they're stuck on 3G and they need to load a webpage that keeps timing out they get frustrated even if they don't know its because of the footprint, it's a poorly made SPA, or whatever.

I think often users do care but they have no meaningful way to tell anyone this or otherwise cause change. I hate the banking web app (from a major US bank) that I have to use. It is incredibly slow, buggy, poorly laid out, and occasionally just decides to put up a spinner forever. Who do I complain to about this. If you call and tell a customer service rep they'll just tell you to refresh and try again, often sympathetically. They know its terrible and hear many complaints themselves, but also have no power to do anything about it.

> they don't care because their _users_ don't care.

I respectfully disagree. I think the users care, but they don't make their own choices - their choices are made for them by people who don't care!

Was MS Teams chosen by their end-users? Nope.

Come to think of it - was Slack chosen by their end-users? No, again.

End-user's aren't given an option, usually:

1. For B2B the choice rests with one (or a few) people.

2. For B2C the choice is made purely because some product got some traction for reasons unrelated to its quality, and that was enough to force the rest of the users to follow or be left out of the network (Slack, Facebook, major shopping sites, etc).

The majority of end-users did not exercise any choice.

Pretty sure we had people spin up a Slack instance at our company while we were still on Hipchat officially. Slack was a big improvement over what we had before that IMO.

This gives engineers a bit too much credit. We have a tendency to heavily over index on last year's problems. That or exploring edge cases that also make no bloody sense.

Indeed, the very existence of so many frameworks is also very easy to blame on errant engineering.

> This gives engineers a bit too much credit. We have a tendency to heavily over index on last year's problems. That or exploring edge cases that also make no bloody sense.

When I was in university (a long time ago, shortly after the big bang :-)) the informal motto of the computer science faculty and students was Computer Science: Solving Yesterdays Problems, Tomorrow!

Now that I think about it, I don't find it that funny anymore.

>> It's the disease that has consumed software

This is another example of Scott Alexander's Moloch: https://www.slatestarcodexabridged.com/Meditations-On-Moloch

"companies in an economic environment of sufficiently intense competition are forced to abandon all values except optimizing- for-profit or else be outcompeted by companies that optimized for profit better and so can sell the same service at a lower price."

I don't think software used to be more secure. As computer users we were more trusting, if not to say naive. As more machines connected to networks, we learned a thing or two.

Open access by default. No passwords [1] or short passwords. Then insecurely stored passwords. Everything in plaintext. Input sanitation? Why bother, only I can input data, I trust myself. Don't get me started on telnet.

I suspect that's another reason why software is more bloated. We started noticing things, how they interact with each other. And once you see something, you can't unsee it. The edge cases we have to account for are growing, not decreasing. There's more hardware to support too.

I'm sure the whole process of writing performant code can be improved. At the same time the bar is being raised faster than we can (or want to) reach it.

1 - And now we're inching towards no passwords again.

Security is indeed an important dimension because it holds the key (pun) for ever more important applications. I agree that a more detailed, like-for-like comparison of software qualities across decades needs to be very careful. Applications have exploded in all domains. But then also the ranks of software engineers and their tools and ability to exchange best practices has exploded.

The trouble with exponential curves is that a small difference in rates can create dramatic discrepancies.

> If its not done its because there is no money in it. In fact the opposite.

Exactly my thought. Incentives are not aligned. There are industry sectors where performance and correctness have value. If you care about the software craft in the same way as the aithor (as I do!) then the best way to enjoy work is to move to such industry sectors.

Care to provide examples of such sectors? Thanks.

Anything involving hardware.

Except car manufacturers.

What makes car manufacturers different?

Not PP, but I used to work for an automotive subcontractor company, and I've heard a few stories about fatal car accidents that lead to lawsuits, which proved that the car design was the reason for the accident, yet the car manufacturer just payed "damages" to the relatives (or settled out of court, maybe) and never bothered to change the design. Apparently, it was more expensive to reconfigure their production pipelines than pay for an occasional death.

That said, this is probably what any big enough company would do. So your point still stands, maybe car manufacturers are no different.

The infotainment system, it’s basically as safe as a web stack (and on the CAN bus).

The only one I have hands-on experience with is financial trading. My workplace is expanding into a new asset class and we're using Haskell for this. The company next door from us is a crypto-trading company using Rust.

I can only speculate about other sectors where it might make sense: - Hardware synthesis (i.e. the software used to design silicon chips); - Aerospace; - Defense; - Smart city devices;

>> There are industry sectors where performance and correctness have value.

> Care to provide examples of such sectors? Thanks.

I worked for years developing munitions control software.

Never once killed anyone by accident.

Not GP, but IME researchers often run big, complex simulations, and often have to care deeply about performance. Correctness, less so, unfortunately. A lot of research software is written by researchers themselves with very little understanding of good software development principles and practices, and end up messy, overly complex, and buggy. That said, being a software developer within research can be the best of both worlds. You should be able to demonstrate (using perf tools, and comparing with existing comparable software) that you're squeezing a lot out of the hardware, while being able to construct maintainable, tested software.

Quite simply: you don’t ship code, you ship features. You don’t ship automated dependency injection. You don’t ship elegant abstractions. You don’t ship cool compiler tricks. You ship new stuff for customers that they’ll pay for. Your high unit test coverage that makes any refactor a painful slog gets in the way of shipping features far more often than the TDD zealots are willing to admit. Most of the “best practices” in the industry is unrealistic dogma created by people in post PMF, entrenched companies that no longer need to do much to print money.

That's how MBAs talk, not how engineers talk.

A conflict as old as time. Unfortunately, it's the MBA thinking that pays the company's bills.

An ideal world isn't one in which either the MBAs or the engineers win. It's one in which they coexist and find a reasonable balance between having more useful features than the competition and not expending too much effort to build and maintain those features.

No, it’s how anyone that’s actually worked in an SMB / non F-1000 / non household name company talks. Most “regular” companies need to focus on getting features out the door.

I’m a software engineer, not an MBA.

You are not anything until you act as something. Acting like an MBA makes you virtually indistinguishable from one. I don't care what you studied, it's what you're doing (or not) with it.

Your entire stance is no true Scotsman with ad hominem. You realize almost every YC company operates in the way I described until they become entrenched in their domain, right? Do you think pre PMF companies are bickering about unit test coverage? If they are, they’ll fail

It's really a basic point that I can call myself whatever I want, but if I walk and quack like a duck, I'm a duck.

YC is a VC firm which is a very specific context that demands things like revenue and growth to be so prioritized as to be implicit and core features of the working ideology. That's not really a good model for software engineering when it comes to social benefit -- it prioritizes something else. It can demonstrate incidental social benefit but that's not actually an incentive that's built into the system that YC operates in and reflects internally.

There are Scotsmen out there, they're just not part of this discussion. That doesn't make anything I said a "no true Scotsman" argument.

Here's a repo for you with no test coverage and no auto-generated DI. They're using unsafe pointers all over the place, too!

https://github.com/id-Software/DOOM

Shall I prepare the postage for the letter in which you'll call John Carmack an MBA? Should we send another to Chris Sawyer? I heard he didn't even write a formal design doc for Roller Coaster Tycoon!

You've either intentionally cast my argument as something it's not or you need to read a little deeper before replying.

Nowhere did I say sloppy code is the problem, what I said was justifying it in terms of profitability is the problem. There's a difference between cranking something out because you're excited to show it and rushing through an implementation because of this or that externally defined money-based KPI.

Engineers understand that the entire discipline is about tradeoffs. It would be extremely easy to build reliable, secure, performant, robust, etc software given infinite time and infinite budget. Engineering is about working in the real world, where you don't have infinite time or budget.

Is it correct to build a piece of software that runs 50% slower, but can be built in 6 months instead of 12? The answer is "it depends" and good engineers will seek to understand the broader context of the business, users, and customers to make that decision.

> Your high unit test coverage that makes any refactor a painful slog

Huh? Unit tests are critical to be able to perform a major refactor without breaking everything.

Only when your refactor only touches business logic underneath the public API. If you do a true rewrite that breaks tests, getting your coverage back up to what it was before the refactor just delays your ship date. And many engineers WILL get green bar syndrome once that coverage % goes down, losing sight of real business goals (staying alive and making payroll). I’ve seen lots of codebases where obsession over unit tests lead to tests that were orders of magnitudes more complex than the system under test. This is incredibly common in multiple process or multi device software, where there’s some sort of custom protocol over an IPC or network or RF layer. You’ll invariably see massive amounts of effort put into mocking complementary parties inside a unit test framework, when the sane thing to do is to just write a 5 line bash script for an integration level test. Or — brace yourself — just do a manual test and ship it today.

Eh, not breaking APIs other teams or customers depend on is very very important. I think broad test coverage is justified there.

Of course there are always exceptions, and testing philosophy should bend to the goals of the organization, not the other way round.

And if a five line bash script is a more suitable test than a unit test in the same language as the application, then that's what you should do. Assuming you have a good way to automate it as part of an automated build and deploy process, of course.

i think ordinary people frequently feel frustrated with low-quality software, but it seems to me not necessarily in a way they can consciously articulate. that is, they probably can tell when software is frustrating to use, but they don't notice a clear difference between:

- slow, high-latency software

- poor resource usage slowing down the whole machine, including from unrelated software

- bad user interface design

- bugs

- intentionally bad/manipulative patterns

if the customers can't even really perceive what it is they are buying, it's not surprising that market forces aren't solving the problem.

i'm not a user interface designer or researcher so of course this could be totally wrong -- just an impression from informal observation

your comment made me think that despite the main thrust of my comment, this trajectory is not completely money driven [1]. the incredible journey of the hardware side of technology (cpu speed, memory size, network bandwidth etc) has also played a role in this profligacy because it can cover for a lot of less optimal design

this is something that happens more widely in the use of resources: you build more highways and instead of relieving congestion you get more people commuting

[1] my open source and volunteer-built linux distribution (will not name names) routinely (like almost daily) prompts for GB sized system updates.

I'm not sure how well "ruthless market forces" are allowed to operate in software. You definitely get some disrupters but for the most part I think a lack of competition is part of the problem.

Microsoft Teams is one of the most buggy, unreliable pieces of software I know, but it's still the market leader in terms of share because of Microsoft's near monopoly of the office suite software world.

It may be worth considering that the market is not for messaging and workspaces, but mainly for integrated business communications systems. If the integration is more valuable to customers than Slack or Zoom's smooth operation, Teams is going to win out.

This is why we need stronger anti-monopoly enforcement.

At a data-centre scale they might still. Even marginal gains in efficiency translate to large savings in power consumption, heat management, and waste. If tech companies are properly taxed and regulated for these externalities, like say consuming billions of litres of fresh water during a drought in order to train a ML model, there would be a lot more pressure for this sort of efficiency.

This is where I normally end up in the software is way too slow today discussion.

Everyone claims to care about the environment until it impacts them even slightly. It's not even hard/time consuming to write good code. People are just overwhelmingly selfish and lazy.

Software devs consistently contribute to the environmental problems that many software devs then complain about and pretend is only a redneck issue. The reality is that we have an incredibly serious idiot issue and there isn't a solution due to the scale of the problem and the corruption preventing meaningful change.

An article the other day pointed out the problem well as well. Though it was related to the increasing lack of people who understand the code underneath the current flavour of the month bullshit framework.

> Software devs consistently contribute to the environmental problems

It doesn’t help that schwabists try to provide indexes such as “Java consumes more than JS”.

This kind of sentences, measuring the immeasurable like a square meter takes more energy than a liter, communicates “We’ll dominate you with power while understanding nothing of what you do, and tax you for using Java”. I bet it’s Facebook’s PHP lobbyists who came up with that.

If heating is a problem, then by all means, tax CPU time. But I already know the tax won’t matter because software provides such an immense value to society. We already pay average engineers $800 a day! But we shouldn’t try to get rid of a language.

If they knew what they were doing, they’d certainly ban NPM. But of course there are already GitHub actions and Sonar extensions publishing the CO2 consumption index of Java programs...

I wouldn't take the approach of banning languages.

But certainly Microsoft shouldn't be allowed to carelessly use a public, scarce resource like freshwater on such a scale as they did during a drought in order to train a model[0] they didn't have a use-case for yet? They should at least pay for the use of that resource to help the local community that now has to deal with the consequences.

The company claims it will "replenish more fresh water than it consumes by 2030," some how... but that's too little too late. And whose to say they will keep that promise? There's no consequence to them if everyone forgets about it and they never follow through. No better time than the present to course-correct.

Resource-extraction companies are like this too. They'll exploit the environment and pollute like crazy unless they are forced not to. They will fight tooth and nail to avoid regulation since that slows or limits profits. But they don't care about people or the environment. Neither do technology companies.

Small gains in efficiency at the level of user-space code has been shown to translate into marginal gains in profits in domains such as ads and securities trading. Data centres also look for efficiency gains here to reduce their operating costs where possible. However I suspect we're getting close to the limit here in terms of gains and motivation. There is still a significant amount of, "throw more machines, air conditioning, and fresh water at the problem until it goes away," kind of thinking since the environmental impacts of those decisions have no consequence to the ones making them.

There's no need to measure the energy usage of the JVM if it's doing useful work. However there probably should be more guidelines for where it's appropriate to set up data-centres and taxing them for the resources they use and pollution they create.

[0] https://futurism.com/critics-microsoft-water-train-ai-drough...

yep. but also mundane and cyclical factors like the much higher cost of money after decades of ZIRP are likely to "encourage" people to do more with less.

There's hardware bloat too. Old fart example: In the days of analog SD TV, the "foolproof" way to feed video into your TV was an RF modulator. The "proper" way was via direct video input of some sort. The "even more proper" way was an RGB type interface.

Except by the end it hardly mattered any more. RF modulators and tuners had gotten so good, that perfectly adequate video resulted from the RGB -> composite -> RF -> composite -> RGB chain. Bloat, but who cares?

In an automobile, the "proper" way to charge a phone is to have a 12VDC->USB type of adapter plug. The "bloat" way is to have a 12VDC->120VAC inverter and then plug the phone's existing wall charger into that. More circuitry, but it gets the job done and it's cheap.

If you like designing electronic gadgets, the "proper" way to flash an LED is two transistors and a couple of resistors and capacitors to build an astable multivibrator. The modern way is to program up a small 8-pin microcontroller. A CPU running thousands of lines of code just to blink an LED? Who cares.

If your computer/tablet/phone are reasonably recent it's the same for software. It's only when your gadget is a few years old that you really see the bloat as formerly performant software "degrades" in later versions.

"Except by the end it hardly mattered any more. RF modulators and tuners had gotten so good, that perfectly adequate video resulted from the RGB -> composite -> RF -> composite -> RGB chain. Bloat, but who cares?"

Eh, no it didn't. That would just look horrible.

This was with a DVD recorder unit. Composite or RF? It really didn't look that different (and both looked good, by SD video standards).

I think software is so complex that the house of cards eventually falls and ruins the company. Then everyone is mad and the workplace becomes toxic. Most companies fail so eventually the market does rectify it but it can be a long slog until that happens, at least when there was so much money being thrown around.

But the quality of the codebase definitely had less effect in the company’s success than you’d expect, like you said. The costs are just shuffled down to the customers and developers while everyone else gets rich.

> ...with multiple decades of massive experience...

You're assuming that the people with this experience have a) successfully passed it on, and / or that, b) they're still coding.

We don't have an apprenticeship model in software development that would lend itself to such a thing. Each new crop of developers has to relearn the lessons in whatever new stack they happen to encounter the issues. It would be like each new generation of carpenters having to relearn their trade because the wood had entirely different characteristics every time someone went to use it.

That experience you mention applies to the general cases, but again, the people that have it may not be the ones doing the work hands-on any more.

> VC funded startups must get to market first or die. Fake-it-till-you-make-it is their religion.

Sure, that's valid. Though most of the world is not comprised of startups. That's mostly a USA thing with a few small exceptions out there as well.

> How come "ruthless market forces" don't rectify this wastefulness?

They do, though people mistake those forces for "incumbents with enough power to influence the market forces". So we're witnessing the reality which they dominate.

I had plenty of examples in my career where better code helped the company long-term but as we all know, there must be a leader somewhere who understands the tradeoffs and give a green light every now and then.

Ask yourself whether you want to be remembered for selling millions of copies next year, or leaving behind something that in time might be cherished by generations.

Look outside software and see what things has been deemed quality and why.

Usually the people doing quality make very few comprises and often they don't do it on purpose.

Quality solely starts with yourself. Only you can guarantee within your own merits and experience what quality is.

Explaining quality is thereby difficult because it is so determined by the personal traits and experience.

My personality trait: I need to feed my family.

Many of us have asked themselves that and the answer is clear: we want something that will truly last.

But short-term interests go explicitly AGAINST that. If I start demanding +20% more time to deliver quality the company will likely start a campaign to replace me in 3-6 months.

From one age (or burnout quantity) we prioritize stability. Ironically it's only then when we can truly provide the good quality but by that time we're much more risk-averse.

sighs

Oh well.

I agree. And this is why I believe most of the points made by Jonathan Blow don’t matter.

And the answer is the same as the answer has always been. When the market fails, you need to look to law and policy. It's really that simple.

We've done it a bit, Apple's little wrist-slap for slowing down the iPhones, for example.

We just need more of it. I know it's all anti-Libertarian or whatnot, but "more regulation" has worked quite a bit in the past and present. Just do that.

If the goal is to make money, you will make money. If the goal is to make quality software, you will make quality software. Sometimes those two goals are in alignment, most of the time they are at odds.

in my experience, when it comes to picking two out of: quality, cost, or speed to delivery, businesses always choose speed and cost. I dont' like it, but I just grew to accept that. The reason why physical engineering seems focused on delivering quality is because of strict regulatory oversight, which for better or worse, we lack in software.

good quality software (say, based on well designed, documented, tested etc. building blocks) can lower costs and improve speed to delivery in the longer run (less need to refactor, fewer bugs, easy to reuse, extend etc. etc.)

the trouble, empirically speaking, is that this "longer run" is not close enough to weigh on decisions :-)

ZIRP is how come the market forces don’t rectify it. Market is mis-allocating capital on purpose, result is “Zombie firms”

Honestly, with some adjustments, the same incentives also hold true in every industry.

I would personally put the low quality of our code more down to immaturity and lack of tooling.

Nah, for 98% of my career it has been managers. I was not always a good programmer, obviously, but even with that, most of the crappy code I wrote was attributed to having to rush.

Reply on news.ycombinator.com