Another Week with Bad Software
thoughtspile.github.io
thoughtspile.github.io
I've often had a contrarian view on this in that I find it almost miraculous how robust software is despite all the flaws in the humans creating it and the insanity of what is actually happening physically to make it work. The fact I can type this out and send it and it will work at all is truly stunning. The fact we live in a world where we are surprised when complex, interconnected, global-scale software breaks and not surprised when it somehow works properly speaks volumes.
That's not to say that we can't do better, but the evolution of software to me has been one clearly of improvement, not stagnation or regression, all while taking on more responsibility, more scale, and more problem complexity. It's easy to point back at simpler times as "the good old days when software was well engineered" but it's easy to overlook the daily wonders that we take for granted today that were unconcievable just years ago.
For the comedic view on this: "Louis CK Everything Is Amazing And Nobody Is Happy" https://www.youtube.com/watch?v=nUBtKNzoKZ4
I'm not asking for hand tuned assembly everywhere ladies and gents, just get within 10x of what the hardware can do before you ship, please! Even if you think "oh its just some chat software who cares" I care! The waste adds up.
Little sad story: my zenbook's SSD died 4 times; the last time it was past warranty and they wanted $250 for the replacement (proprietary port). Got a zenbook / sata switch from aliexpress, never buying macbook copycats again.
Why? No mitigations/redundancy/workarounds and spoiled plans.
Example. My last job had mostly automated provisioning process and you could not do anything for the first 3 days. My current job is at a smaller company that has manual provisioning and you can get started within first couple hours.
Some automated systems completely lock humans out of the process, so it turns into Brazil.
Simply planning the route in advance with path finding or something more? Garbage collection was around and not very different before software was prevalent.
No further Requirements. As many or as few paths as you'd like, no worries about efficiency, sure you can go through the same streets many times if it helps. But needs to cover every street, obey orientation, and be readable by others.
Does not sound easy, does it?
Now, let's add a few more Requirements...
So yes, "one thing" might go wrong out of the 10,000, but that "one thing" can cause a literal panic attack. I don't have sympathy for companies that create crappy software that barely limps along and causes a lot of distress to my wife over stuff like that, or not telling us we need to pay extra to book seats on an airplane in advance to be able to sit together for a 13 hours flight. No sympathy at all.
So, all things being equal, price beats good software...
However: the use of software and the dependence on software have increased so much, that according to my guesstimations it's almost guaranteed for someone to encounter a software annoyance or have a software-related incident perhaps at least once per day and be involved at least in one major software-related screw-up per year.
And these numbers will only go up as software systems become more connected and intertwined with our lives. From that perspective, software in general is not good enough and people are correct to complain.
It's always easy from the outside. Not so much if you are in the trenches yourself.
There was a time, oh what a time, when I could use hotels.com and booking.com to get really good prices and still stay at very nice hotels if I was ok with booking close to the date.
Remember a time when a friend and was given a literally shitty room (with overflowing toilet) in Cairo via hotels.com. Hm, doesn't rime well with the paragraph above. :)
Now that I think about it, Quicken and TurboTax too. People signed up for all kinds of important things using my email.
This was all from their HR person on the phone.
They offer more money than most other companies in The Netherlands .. and I think it was clear why. ... I passed.
I think they're doing only light testing for features before they push them to production because they're doing so much A/B-testing that all small "improvements" that decrease conversions or $$$ per minute are pulled again very quickly. So it sounded like a lot of small PoCs that then get rewritten properly when the idea is good.
And the underlying meta problem is that the people who are supposed to be responsible for figuring out which is which haven't got the faintest idea how to tell. Compound this with the fact that most management structures are still based on factory-assembly floor management where figuring out who wasn't turning the lever as fast as they guy next to him was the most important metric and you end up with corporate structures where trying to figure out which developers are better than which other developers actually has the opposite effect than the one they want.
In my experience as a developer, we've never been given the necessary resources (time, QA access, specs that don't change every day, sample data, redundant hardware, etc) to build a reliable system. Managers always pushed me to have something that looks mostly-working, and shipped by the artificial deadline, and argue that any known bugs or limitations are not important enough to fix before the next round of new features.
It's not so much that I'm not required to care, but that I'm not allowed to care. I get better food at Canlis than McD's, too, but given the time/ingredient/recipe constraints, even the Canlis chef would struggle to make something great if thrown into a McD's kitchen.
This was my point, mostly. It's not a problem with developers, it's a problem with business priorities. I agree with your experience.
And, just because someone "thinks" they can manage, doesn't mean they can manage.
The only metric that matters: number of USERS. Good developers have thousands of them. Great developers, millions.
More to the point: everything’s well and truly fucked, yes, but surely more education cannot form the skeleton of a solution when the masses we’re supposed to educate don’t actually want to learn anything. IMHO, the very incentives of business need to be changed as they’re the foundation of every bad decision that manifests as software bugs. Is this not a version of Conway’s law? “All bad software reflects the lopsided incentives of the ambient business environment.” Call it the universal law of fubarness.
Yup. What's worse is how many of our own peers fall back on justifying all this crappy software with "businesses are for making money" without a single dose of irony. It was a bit depressing, reading the comments on HN on the other post and realizing how few people seem to actually care about writing good software.
> the masses we’re supposed to educate don’t actually want to learn anything
Indeed.
I can't remember which thread this came from, but this talk was posted here somewhere on HN recently and I found it pretty interesting and relevant to this sort of thing: https://www.youtube.com/watch?v=TGGGDpb04Yc
If "good software", whatever it is, is not economically viable, I would gladly walk away feeling good about the future and open a beach bar instead. Currently it seems that faulty design and code are actively harming the businesses. I personally need to make more research around here — what's the connection between, say, conditional spaghetti and anything important? We should at least start talking about these implications. No code is an aisle)
Same goes for "the masses don’t actually want to learn anything" — I see many people getting into web dev right now, and we've made it unreasonably hard to do — why do simple (as in, the ones you can make after a bootcamp) web pages look and work so poor? Word documents from people with a week of training look pretty reasonable.
Good software is exactly economically viable, but it takes upfront investment. Unfortunately, businesses don't like to make that upfront investment in something they don't - yet! - understand. Think of a car: a young person might treat their first car like crap assuming that it would simply run forever like magic. This attitude does a quick 180 the day the car's tranny drops out. I'm thinking that not enough software "transmissions" have quit in order to instill these lessons within the MBA-s of the world.
The supply / demand gap on programmers should close once — our job is not inherently bad or risky. I feel that we just don't know how to train new programmers on this scale yet.
So long as a human exists to work around the faulty code, the harm to businesses is minimal. For example, many companies sell feature-full, but absurdly slow and buggy, software to other businesses. The customers aren't going anywhere, and they've protected their niches with a minefield of patents.
So, the features hook the execs, and the end-users work around the bugs and slowness, the business sees no blowback from this.
My favorite anecdote was a front end engineer claiming "I went in and profiled our code and made an action go from 5 seconds to under a tenth of a second!" To which I can only go "Wait, there was a user-space action that took 5 seconds?!"
And I'm guessing there was, and you didn't notice it (or stopped noticing it at some point), because that's what people do. They get used to this nonsense. Especially if they have no power to improve it.
If it turns out the action is more useful than originally planned, you should take time to optimize it.
For programming, I enjoy https://igoradamenko.com/blog/ These seem mostly dead, but might come back: - https://medium.com/@kovchiy - https://medium.com/@xanf Here's a guy working on V8 who ocassinally posts / talks in Russian: https://mrale.ph/talks/
(as for in english, i've rarely seen people talk in other languages, i thought maybe there was a rule)
80% of the time, you don't.
Alas, which leads to this: Programmers are often the worst kind of User.
Anyway, got everything set up. Everything working fine. Rebooted a few times: no problem. Popped in an extra hard drive which is just for file storage for Steam games. Reboot and... nothing. Pulled the hard drive. Nothing. The operating system is now in a completely broken state. No start up messages, nothing. Not even NumLock on the keyboard works.
I have no idea what happened.
Disclaimer: been running Ubuntu Studio for 10 years, and it is still the #1 music-/audio- processing system in my studio, which of course has great MacOS stuff too, but still .. UbuntuStudio .. just freakin' great audio, on Linux. Suspends just fine too.
(Clue: its jackd. I nuked pulseaudio from orbit.)
- Visual Studio 17, crashes, has bad UX (for example try to find a symbol and it pops up a modal dialog). Refactor tool works about 3 times and then fails with "blal bla (null) bla blah" error.
- Win10 explorer is often confused, starts to render the start bar wrong
- UE4. Oh god, I think every single feature in it has at least 5 bugs.
And don't even get my started about the state of software on mobile phones.
I see bad code everywhere. There are fundamental issues that I see repeating and typically these are very cluttered designs muddling several things together. Clearly the developer(s) didn't have a clear idea of how to design software and abstract it into pieces that are independent and reusable. The way they are they're hard to design, write and debug. Nearly impossible to unit test and practically impossible to re-use. All these bad practices and downright incompetence of many developers must be costing this industry as a whole billions of dollars.
- Overlay the huge "you need some updates" panel, with no option to close the window. Every key input brings you to the update installer.
- Windows Defender frequently ignores my exclude lists, which causes it to start scanning my VM's .vmem files, reading and writing several megs per second to disk, causing my VM to be unresponsive for 20+ minutes if I do not restart my entire system.
- The new search options in the task bar seem to have gotten worse since Windows 7. Whenever I start typing something like 'not', it shows me Notepad++. When I then add the 'e', the options disappears.
- Lately, my Windows 10 has decided whenever I boot up a system, it should restart all processes that were active when I shut it down before I have had time to even log in, which is kind of a fucking bother when I had loud music playing in the background when I shut it down.
- All the goddamn bloatware on it. I know that when I buy a DELL or HP, I'll have to completely reinstall the system to get rid of the bloatware, but Win10 comes with its own set of it for some reason.
- All the 'smooth' animations, which slow the UX just enough to be frustrating. Don't slowly slide up the start menu, just pop it into place!
Let that sink in for a minute. Microsoft has somehow managed to screw up a goddamned menu so badly that it can break in multiple ways they don't know how to fix, and rather than fix this bug they have devoted their resources to bringing us features we never asked for.
That's today's Microsoft. Adding (broken) new things no one wants, changing things for the sake of change (see: new settings panels), and never fixing any bugs. It's as though they've embraced the worst qualities of open source developers, only they charge you for the privilege of using their OS and still have the gall to force ads on you.
--
[0] - half-exception, because while it made the desktop experience worse, it also enabled Windows to work well on mobile devices, and it turns out Windows is the only OS for tablets that's not a toy, but a tool for productive use.
A good trick, considering Windows 10 went RTM in 2015!
PS: Seems it was because of this: https://answers.microsoft.com/en-us/windows/forum/all/start-...
which appears to be a similar problem on Windows 7.
I haven't seen the full screen update modal recently, though. I thought it had been removed. Maybe it was only removed in enterprise?
Maybe I'm just stupid, but I refuse to waste 2 hours of my life googling what has always been a relatively straightforward task. And it seems that every UWP program has to have ~40% of its "advanced" features removed like this in the name of ease of use.
Is this not often the case because you have people that are not actual developers involved in the process of software creation? In the sense of, you don't get budget to fix/refactor something because business says so; you have to make your app comply with some shitty legacy standard because business wants to have your software work with some stuff from 20 years ago - and so on, the list is basically endless.
Don't get me wrong, developers (myself included) are probably more than enough at fault themselves, but if 'creating software' was only about making nice, clean, awesome code that really does what it's supposed to do, devs in general would probably make a better work out of it compared to what is happening in the real world - where non-dev people also have a say in what/how you create software.
Which one is that? Ctrl-T ('Go To All') isn't modal. And it goes to symbols only by prefixing the search with #. So it might be a better fit than whatever you're using?
You say that but big companies like Amazon can calculate how much money they're missing per second of their services not being available, or how shaving off x milliseconds off of each request reduced their cost by millions per day. They also advertise with their uptime guarantees to haul in big contracts.
Disclaimer: I am in risk management for financial infrastructure.
Lest, e.g. some Amazon deliveries don't come in 1 day but take a week?
Or some employee doing some BS job can't use Gmail for a couple of days?
Like thermostats. I've read about Nest thermostat crash on loss of Internet connection which result in dangerous temperature loss in infants room. We can make those thinks more robust for years, but it does not have an app and does not make so much money.
Food supply was original comment. Most people do not have food for week. (I compare it to always stocked pantry of my grandparents). Any small - one week - disruption can result in big problems for some.
But we are moving to hi-tech just in time system everywhere, which are dependent on Internet, GPS, advanced forecasting etc. Those systems can be for example hacked and not functional for days.
I don't care about Amazon delivery being late.
The best thing to do with them, and most of this other godawful "smart" home tech is to rip it out, smash it with a hammer, and put back the cheap analog versions you had in originally.
But mine has not outright crashed when its Internet connection was interrupted.
I'm disagreeing with, or pointing out, the incorrect use of the word "dangerous" in order to get clicks for an article.
I can see the harm in frozen pipes and whatnot, but you take the same risk when dealing with other systems, such as other thermostats or power or the furnace itself. However, if it is known that Nest devices fail when the internet fails, then that is Nest's responsibility to advertise that to users.
I'll grant you Amazon, but turn off GMail for a couple of days, and you'd dip the economy. A great many of small but essential companies - from car workshops to paint shops to furniture makers to retail and wholesale shops - run their business on personal GMail accounts. Losing access to primary e-mail for a week may very well be an existential threat for them. One which they don't realize, because GMail is generally reliable.
Yeah, but we've managed that in the 60s and 80s without so much as the web. Trucks are still needed of course.
If there's anything that could make the software better, it has to be an economical incentive. Learning to build reliable real-world systems that rely on faulty software (you might call it IRL erlang) would be great, too.
I know you probably mean this for software, but it's not limited to just that, and it's as ancient as humans are (or animals, to take it further, they make mistakes as well from time to time). I mean, instead of booking.com messing up (or whatever it was) I've already arrived at a hotel once just to find out there was no trace of my booking. Someone lost the paper or forgot to write it down or so. Turning the human off an on again, a.k.a sleep, also helps with this from time to time and makes it a nasty hard to reproduce bug. And hard to fix as well. I get the analogy only holds - usually - when it's not about bugs which are reproducible time and time again. But on the other hand those are usually easier to discover and fix so maybe less of a problem. tldr; not sure how worrying it is.
I guess (hope) avionic / nuclear software is generally of higher quality, but must it really be so hard and expensive to develop well?
Actual, robust software takes thousands of hours of TESTING, nevermind development. Multiple vendor cycles. Field studies. User observation to limit system error.
And unfortunately (if you're not a programmer) programmers are very, very expensive, but most software must be very cheap. So unless you're at the largest scale, your software is likely crap or very expensive, because the market won't support anything else.
Personally, I had interesting booking.com situations too. Like, I arrive to a booked apartment, dial the password, the door opens, I see lots of Chinese people who arrived just before me. 2am.
Haven't had a single problem yet. Not a single one. I'm not saying it doesn't happen, but...
...I suspect the hotels are declining your business and using "bad software" as an excuse. For this to happen twice in a row is definitely suspect.
Which makes me wonder how many other things that we blame on bad software, are actually socially caused?
Interestingly, there’s curently another article trending describing the concept of Poka-yoke [1]
When a hotel doesn't have a 24/7 online system like the above companies, booking.com simply relies on the hotel to update their inventory on booking.com's system manually, and then faxes the hotel everytime a reservation comes through. The fax can fail, and so can the minimum wage employees who may or may not be tasked with updating booking.com's systems.
Bottom line is always confirm, and re-confirm when the consequences are great enough to warrant spending time re-confirming. And especially try to deal with the vendor directly, onsite especially. If you make a reservation at a hotel that doesn't have an app that shows your reservation in real time, then call and get the email address of someone who actually works on site, and then email them to confirm your reservation, and so you have something in writing from someone at the hotel.
Some of them will refer you right back to booking.com though if you want to book online, in which case it's probably fine and the hotel uses booking.com directly. Or booking.com sends them a fax or email.
I just confirmed the booking by paying. I shouldn't have my booking cancelled because I didn't respond to an email. We can do better than that.
Maybe, as others had already mentioned, I ignore them because of all the overwhelming times that software worked amazingly well for me.
Anyways, I've just fixed it (might take some times until the cache expires, don't bash me over again ;-), thanks for the reminder!
The default Jekyll setup does indeed rely on some implicit theming, which is not very convenient.
If I were to already have the artifact files locally I wouldn't commit them in git and push to GitHub, I'd upload them straight to S3/Neocities/whatever. The whole point of Pages is to push some source material and have some CI/CD do the build and deploy.