A person after my own heart.
I've had many a dev go "why would you do that"
In which I answer "it doesn't matter, but if you accept my input it's your job to ensure the app doesn't crash"
Then we hired Kevin.
Kevin had the handset for 40 minutes before piping up "crashed it". The lead comes over to have the sequence explained to her, and says "huh, nice edge case". Half an hour later "crashed it again" (in a completely different way). Explains the sequence to the lead again. An hour later this happens again and he explains the sequence and she finally bursts out "Why would you even do that?! How did you think of pressing those buttons like that with that timing?!!".
Good testers just think differently than software engineers.
There aren't any bugs as long as I don't look for them!
I'm sure in some cases it's useful to detect all possible crashes, e.g. to make an app as secure as possible. In other cases I'd watch out for diminishing returns; perhaps instead of "think differently than software engineers" it would be enough to "think in the similar way as product users".
Software engineers tend to use products in a consistent way (based on how they know it's meant to be used) whereas good testers explore the space of possible inputs in a much more 'creative' way.
My point is that the extent of testing should depend on the actual product; eliminating all bugs is not the primary goal of every team.
Some companies might decide on other goals and prioritize e.g.: just paying users (giving them better support to resolve issues), or acquiring new users, or something else.
In a similar vein, I've also used U+2212 as minus sign in my regional settings. There's a lot of software that refuses to parse numbers it previously happily emitted.
I've given up on that too, by now, though. The only thing I still do is using English as UI language (so I don't have to deal with bad translations of software), but German as my regional settings (with ISO 8601 dates). There's a lot of software out there (I think GNU gettext is broken in that way on Windows) that assumes that the way I want my dates and numbers formatted has any bearing on the language I want to see in an application. Many others don't care about the regional settings and use the UI language to also format dates, times, and numbers. That's annoying, but at least nothing breaks, so that's the only deviation from the standard user I still use, to still be able to work.
I write a lot of javascript and the string "null" is pretty harmless in most code. But there's all sorts of fun bugs (and often security vulnerabilities) you can find if you make an identifier "__proto__". (If code ever uses that as the key in an object, you're off to the races!)
If I pour water in the gastank of my car, it will also fail to drive. Or gas in the sprinkler tank. So the car should somehow prevent the enduser putting the wrong thing in the tank?
I'm confused why the parent comment is being downvoted. It's a valid question. It might sound naïve to some but it's still worth discussing.
The analogy would be something like that:
- if I throw spaghetti on my windshield, my car shouldn't break down
- if I hold the wiper's stick to the position that runs it once (instead of putting it in the position to continually run) my car shouldn't break down
Because software by virtue of not requiring physical access is much easier for bad actors to mess with.
Abuse of such also seems to be classified very differently than abuse of physical systems by human brains e.g. almost no rando would think of putting sugar in your gas tank while walking near your car, but nobody blinks at fucking with your input fields.
If there was a physical device that could filter gas and non-gas liquids that could be installed in a car we would expect car manufacturers to do that.
I have seen software that puts the onus on users to use it correctly "Hey user, don't enter more than 5 items in this list". Because the software can't be bothered. And if the user enters 6 items in the list and the software crashes and the user loses all their work, everybody can point at the user and tell them it's their fault for not following directions. But personally I'd be embarrassed if that was my software.
There is one thing they could do to stop people putting the wrong liquid in their car. That is to key the nozzle to only fit cars which support their fuel, so petrol pumps nozzles only correctly fit petrol cars, diesel pumps only fit diesel cars.
Completely agree with you on you main point, software should be made correct and resilient (more than other things) because it can be made so.
Your proposed problem will probably only affect bad spellers, who have a break pedal.
Of course if you're trying to break it, everything is possible.
Reason: The gas nozzles dispensing unleaded gas were made smaller to prevent people from putting leaded gas into a vehicle that required unleaded gas (which would poison the catalytic converter). The diesel nozzles remained unchanged and leaded gas (with the big nozzles) went away.
Source: I had one of those “evil” Jetta’s
https://www.autonews.com/article/20130521/RETAIL05/130529968...
So, the security of my house (at least where I live) does not have to be so resilient but the security of much of my software does.
Clicking a button that was enabled for me and does many things in the background is not so obvious.
Also, cars have been accessible to everyone way longer that PCs and smartphones. Most people alive today (in developed countries/areas) saw their dads driving when they were kids. A person in their 60s didn't have a PC when growing up
Then there's just annoying stuff like the case where someone paid to have "null" on their car's license plate, which suddenly caused him to get all traffic tickets that could not be correctly addressed.
Validate your inputs. Be very careful with in-band special values and escaping syntax. Don’t make any assumptions about what is “reasonable” input. If you have to make assumptions, document them and validate all input for conformance. Always check what requirements and preconditions the code you call has on the values you pass to it. Don’t just make assumptions about it.
But I'm pretty sure the manual say you shouldn't do that. Like microwaves say you can't put living animals in it.
First off, we expect all things to be as resilient and reliable as makes sense via a cost benefit analysis. If it's cheap to fix, and expensive not to fix, we expect it to be done. If it's expensive to fix and cheap to ignore, we expect it not to be done. And of course, if it's impossible to fix, we definetly expect it to be ignored. :)
So, we expect that cars should NOT catch fire when rear-ended, because it's possible to design them not to do so, it's not that expensive to design them not to do so, and innocent people could be seriously harmed through no fault of their own.
But water in a gas tank? I can't think of a way to stop someone doing that. And it would just disable the car if you did it. And since cars have locking fuel tank covers, you're really limited in your ability to maliciously harm other peoples cars.
So in the case of cars "explodes when rear ended" is not okay but "stops driving when you fill the tank with water" is okay.
Software, by its design, is often more fixable than other things. You can filter the inputs to a log in form whereas you can't really filter "things people might put in their kitchen blender".
Second, note that software is, bluntly, a lot less resilient than most things. I've got a hammer sitting in my garage, and it's just going to sit there until I do something with it. It won't randomly stop working, it won't auto-update to a version that is incompatible with my nails, it won't be remotely hijacked by Russian scammers to break into local businesses, it doesn't need patching. There will never be a CVE for this hammer. :) I've had it for many, many years, and I'll have it for many many more, and it will be just as good a hammer in 10 years as it was when I got it. We can't say the same thing for software. And since software is just way more of a dumpster fire than "normal" things, we have to expect that more work will need to be done to counteract that.
(There is, as always, at least one relevant XKCD here. In this case, I think https://xkcd.com/2030/ is on point. The more you know about software engineering, the more you'll realise the entire thing is held together with bailing wire, duct tape, luck, and an intern trying to live edit the production database to fix the data errors before anyone notices.)
Third, and very much related to the last two points, consider the scale. Your car's gas tank, or your building;s sprinkler tank, are vulnerable to various attacks, but it's not vulnerable to being attacked remotely and untraceably by almost anyone on earth via a number of low skill attacks. And of course, software also can yield larger rewards. If the local corner store has a dodgy lock, maybe you could break into it (at significant risk to yourself!) and steal some cash from the till. If you can compromise the head office network of a major retailer, you could steal millions of dollars.
Edit: Also, I'm aware of a nationwide outage for a pizza chain caused by a phonebook. There was an internal webpage that some stores looked at occasionally to show stock levels or something similar. It was quite a slow/expensive page to load, but because it was loaded quite rarely, and only by a small number of internal users, it didn't have a lot of cacheing or rate limiting on it. Someone in one store shifted something on their desk, then walked off. This caused a phonebook to shift, and depressed their F5 key. That caused their browser to start refreshing this page very, very rapidly - multiple times per second. The load from this actually overloaded the central servers, and the entire system went down, stopping orders from being placed or printing out. So it might seem silly to say "hey, what happens when I do this thing I shouldn't do", but actually, over time, all sorts of things that "shouldn't happen" will happen for some reason or other. If the result is that every store of a nation-wide chain goes offline, that is....not great. And software is just way more prone to these sorts of things than others. If you tell me there's a vulnerability that lets people easily open the door of any hotel room at a large chain, I'll instantly bet you $20 it involves smart locks, NOT traditional mechanical keyed locks. (And indeed, that's happened more than once, and it's always been a smart lock to my knowledge.)
It's very crude and not at all foolproof. For the lack of sophistication it's shockingly effective at highlighting a huge amount of assumptions we make about how software is / can be used.
If you tried ctrl+n (I think), you could advance to the next level, but only if you had beaten the current level or had the pass code for the level. In a bout of frustration, I rando-mashed the keyboard and advanced to the next level, and could then ctrl+n to the last level! I could reproduce the effect, but never worked out the actual combination that unlocked it. Good times :)
Fuzzing is a pretty popular testing technique for libraries, but GUI software has not seen the same attention.
Example of tools: https://www.fuzzingbook.org/html/GUIFuzzer.html
Something similar should be reasonably easy to build these days using AutoHotKey or the like. I bet it's been done.
This is how try/catch alls get added :(