Is Software Getting Worse?
stackoverflow.blog
stackoverflow.blog
I think one reason why software is slow because managers don't allocate time to improve performance unless it gets worse than "good enough". For me that meant taking time to improve performance without being told or telling anyone how much time I spent trying to improve performance.
When I used to work on Android in a large company my favorite low hanging fruit was looking where people used `Cursor.getColumnIndex(String columnName)`. People would call it inside a loop for every row in the query result. This was a surprisingly large time sink according to the profiler.
Some developers just don't know or care about performance. I depended on a system that had very large list of banned/blocked IP addresses. This list was stored as a giant array of hundreds of `BigInteger`s. Each time the function was called to determine if an IP address was banned it would convert hundreds of strings into an array of `BigInteger` and then loop through the array checking against each one.
I got to dig in to some hotspots in our code, some hotspots in the test suite code and speed things up significantly. It's incredibly rewarding work. But as the blogpost says, you need to know how things work (and get good measurements) to be able to make a difference. Knowing what tradeoffs to make, such as caching things to trade off memory usage for computation vs reworking algorithms to be more effecient in the first place, is tricky and not something you can necessarily hand over to a junior dev.
It's amazing how performance killing patterns transcend the operating environment. I remember profiling a loop-intensive .NET app many years ago and noticed a massive number of framework-level calls to a getOrdinal(columnName) method to find the numeric index of a column from its name. That was triggered by a programmer calling row.GetColumnValue(columnName) method. Though it was fast, it was called millions upon millions of time. I realized a few orders of magnitude worth of performance improvement by manually getting the ordinals on the first iteration, caching them, and then using them for subsequent iterations.
I love performance profiling and improvement. It was almost never "roadmap" work, but something I just did.
And even then, it's "come up with a quick fix by tomorrow morning to address the problems caused by the mountain of other quick fixes that created the problem in the first place".
I disagree. The reason developers and managers don't care about performance or more generally, good software, is because customers don't care about it. There's no incentive to build more robust future-proof software unless someone is willing to pay for it. Given the choice between cheaper "meh" software with tons of shiny new bells and whistles and better-designed, but more expensive software with fewer new features, guess which one the customer is going to go with 9 times out 10?
It's like how everyone loves to complain about the poor service and general crappiness of commercial air travel. We all do it, but almost all of us also buy the cheapest flight we possibly can and then wonder why the airlines ended up cutting so many corners.
Code quality is a trade off. Sometimes that trade off can result in a ROI by lowering long term maintenance costs enabling you to deliver new feature more cost effectively. Sometime that trade off means that your startup was too late to the market and a competitor was bought by Microsoft instead. Regardless, the trade off needs to be discussed and the options weighed.
I get that programmers want to view themselves as craftsmen making a work of art. I’m sure there are plumbers obsessed about crafting the perfectly soldered joint. I respect both of them for their efforts and passion, but have to acknowledge both misunderstand the role they are being paid for the majority of the time.
For all of Jonathan Blow’s wisdom, I find that too often he’s just a curmudgeon yelling at kids to get off his lawn and waxing poetic about how much better it was in his day. He suffers from the same malady that so many people seem to fall into throughout the generations. That the new generation is shit and everything is falling apart has been a consistent theme for the last two thousand years at least. Quite frankly it’s old and boring.
I used to be able to listen to my music while driving through areas without cell phone service as my entire music collection was stored on my phone. Now I have to manually download tracks even though my phone has 8 times the storage of the older version.
Stripe updated the app recently. It now requires Face Id almost every single time I change screens. Really? How is this "Better" to view read-only information?
I tried to take a photo on my iPhone in the dark the other day. I start by turning on the flashlight so I could see what I was trying to photograph. I go into the camera app and the light turns off so I can no longer see what I'm doing. WTF?
I used to use Gnome for my Linux desktop. I liked being able to change window decorations, especially for the smaller and more compact window titles and such. That feature got removed. Then the Gnome default icons and such kept growing and taking up more screen real estate. At least in this case I've been able to switch to the far less bloated LXDE.
It goes on and on. I feel like the UX designers don't pay any attention to the user experience anymore. Don't get me started on web pages that assume you're on a mobile device when you're on a desktop with lots of screen real estate...
<sigh>
Oh they do, they pay a lot of attention to how many ads the user experiences.
The worst offender here is android. The clock app especially feels like it's been made intentionally to be terrible. For example, when you delete an alarm, the popup "Alarm deleted" appears at the exact place where the button to delete another alarm will be. You have to move the list of alarms, or if you can't, wait. There is no way to select multiple alarm to delete them at the same time. It's a list but you don't have one of the most basic operations on a list, do something to multiple elements. The UI is getting worse and worse, every few month there's something else that will look ugly, like a massive + button at the bottom center of the screen.
I assumed that the reason was essentially economics. In the Old Days, hardware was expensive and programmer time was relatively cheap. The economics were in favor of writing high-quality software. These days, it's just the reverse. The economics are in favor of writing substandard software and relying on hardware grunt to make up for it.
I don't get this. Hardware can't make up for dark patters, awful UI, missing features, bloatware, vaporware, and adware. The only thing hardware can make up for is slowness, but even that is highly questionable: a big hardware upgrade could make a difference of 50%, but improving the algorithm or eliminating a SELECT N can make a difference of multiple orders of magnitude.
Yes and no. The main source of slowness in modern software is memory access (patterns). Yes, a binary search is just O(log(n)) but if your array isn't to big a linear search will probably be faster, just because the chance of your data being in cache is much higher due to prefetching.
But it's much harder to plan, program or to refactor software to have a cache friendly memory layout and access pattern. And good luck optimizing your Java, Python, etc. code with respect to memory when even integers are pointers to some data on the heap...
I don't think it is. Memory access patterns matters a lot when you want to squeeze every last bit of performance, or when you want to update old code that was optimized for different assumptions. But I think most slowness comes from not having looked at what could make your software slow. Maybe you forgot to compile a regex before a loop, maybe you're copying a whole array at each of the 100 000 iterations, maybe your algorithm is accidentally n^2.
If your array isn't too big, it's not the bottleneck. You're talking about tippy-top-tier optimizations to eke out the last 10% possible improvement. I'm talking about the extreme prevalence of bad, lazy, thoughtless programming needlessly slowing down most systems by 100x or more.
Maybe you work in embedded systems and drop into assembly to manually optimize a hot loop, but most of us don't. Most applications in the world send thousands of SQL queries to a database to do a job that should require only three.
Excepting for dark patterns, vaporware, and adware -- all of which I consider "malware" -- the rest is, I think, due to the use and abuse of frameworks and libraries whose entire goal is to reduce the amount of developer's time required.
So they count as cost-savings, and result in poorer quality software. Hardware doesn't make up for them, but hardware makes their use more practical. The additional memory usage, disk usage, and reduction in performance are easier to overlook because the hardware can absorb the blows.
In the old days you had cheap junk products just like now, but if you spent more you could purchase high-quality products. High-quality products used decent materials and were designed to fit their purpose and built to last. You could reasonably expect that if you had any problem and you shipped it to tech support you would get a repaired product back, no questions asked, for free if you have a warranty.
Hence the saying: "I'm too poor to buy cheap shoes".
Nowadays it's different. The Sale(TM) is everything, and the product is the minimum viable piece of garbage that is able to facilitate that. Cheap junk is still junk, of course, but more expensive products are simply the same junk but more expensive and rebranded, with a "high quality" sticker on it.
Some industries are more affected than others of course, but for all of them, if there is thought put in the design, you can basically guarantee that nowadays some of it will go towards intentionally making the product break itself over time so that you have to buy another one.
Go and find a hand blender for less than $1000 which gearing and mating parts can take the load of its motor and not break over time (i.e. it's not plastic). I will wait.
You have a problem with your product? Well the answer is go away and buy another one. There is no support. Whatever semblance of it may exist is there because the company is forced to have it and it's basically low-paid flowchart executors that are too busy surviving to care about you. The flowcharts, of course, are designed to make you go away and drain as little of the company money as possible.
If you send your product back to get repaired you may get it back the same way you sent it. Or you may get another customer's product which is maybe more scuffed than yours. Or you may get charged and have to pay to get it back even if you have a warranty because they say your product breaking 3 days from purchase is "normal wear and tear".
IF you have no warranty expect to simply get rejected or to pay more or less what a new product would cost you, making the whole thing pointless.
It's more likely though that you won't ever find how to send it back because the instructions to do so are as hidden as humanly possible and there is no phone to call where a human will listen and not hang up on you.
Now, it was always more economical to sell junk as expensive as possible. It was always more economical to not give support, to not sell repair parts, and to not give out repair schematics. That didn't change.
But it didn't happen in the old days (at the very least it was not as widespread as it is now).
Because in the old days people Gave A Shit.
(/rant)
I’m not an expert, but maybe theres a lot more that devs have to pay attention to these days than in years before, like how a windows 98 install would just be ravaged by malware almost as soon as it got connected to the modern internet…
When on dialup, even a few hundred K of a web page/site mattered. On a lan, it mattered a lot less and the modern internet is closer to a LAN 20+ years ago in terms of performance. We have computers in our pockets faster than desktops at the time. I had 64mb of ram in 1997, today I have 128GB. We fill the gaps as technology expands and will generally look into issues as it's necessary to support new platforms (arm, risc-v) or simply because it's too big.
We complain when tools like Photoshop or MS Word move to a subscription cloud model, but maybe they're just getting realistic about pricing.
You used to be able to buy these tools for a 3-figure range, maybe low 4 figures with all the bells and whistles. Today, creative cloud starts at $55/mo, Photoshop $21/, MS office ~$6/.
If you use a 10x annual revenue to price these, it'd be something like $6600, $2520, $720 respectively. Would you pay those sums for those tools? Would a starving student?
Software is still hard, and good software is still expensive. Calling it "undervalued" is probably accurate; if it weren't for the downward pressure of piracy and open-source, you have to wonder if we'd have software dealerships selling packages as expensive as today's cars.
I've seen a lot of software pivot away from things and remove features because of metrics. If a feature is used by a minority or less than average, it's seen as useless and removed. For a lot of users, this removes something entirely, which they might have used infrequently but to great value.
Sometimes you're part of a tiny group that uses a feature daily and it gets removed entirely, making the software much less useful or useless alltogether.
I've seen people admonish developers to do this for over 30 years now. I've never seen a developer or group of developers be successfully able to deftly navigate the complicated politics around actually doing so.
There's a big difference between 30 years ago and now, and that difference is that hardware performance increases has slowed to a crawl.
It's possible that in the near future software might start competing on performance again. (I hope.)
As a user of web sites, I agree, but I haven't observed that management much cares.
Somebody needs the guts to say "no" to too much junk in the trunk. I take a scoring hit when I say accessibility often goes unused for small/niche biz apps and thus should be skipped or downplayed in some cases, but it's the blunt truth. ADA laws say "reasonable". The cost/benefit math is often not reasonable in many situations, and I'll take the case to the jury myself if necessary. Ignoring YAGNI is economically costly.
A lot of it is also caused by dev's trying to get as many buzzwords on their resume as possible. Between Resume Oriented Programming and managers' fear-of-missing-out, we have bloated stacks and bloated software.
YAGNI and KISS matter, but ignored by silly human tendencies. I think I'll find a job on Vulcan...