Operant Conditioning by Software Bugs (2012) | Hacker News Reader

Operant Conditioning by Software Bugs (2012)

blog.regehr.org

40 pointsdceddia16 comments

> The second reason I wrote this piece is that I think operant conditioning provides a partial explanation for the apparent paradox where many people believe that most software works pretty well most of the time, while others believe that software is basically crap. People in the latter camp, I believe, are somehow able to resist or discard their conditioning in order to use software in a more unbiased way. Or maybe they’re just slow learners. Either way, those people would make amazing members of a software testing team.

I think this is one of the big differences between junior and senior software developers. You can see juniors get very frustrated with bugs because the code they delivered was 'working'. Seniors realize that software only works within well defined parameters - and it's only a matter of time before you find another parameter that you did not define well enough beforehand.

I also think there's a hierarchy of software implementations:

Working As Expected (by the user)

Working As Designed (by the designer/lead)

Working As Intended (by the programmer)

Working As Implemented (by the programmer)

[this is somewhat tongue in cheek]

  > I think this is one of the big differences between junior and senior software developers.

This may just be the curse of knowledge, but as a senior developer I would have to disagree. I've thought software was crap as long as I've used it, and I believe I can find a bug (or at least what I would consider a bug) in about five minutes using any software. Part of the problem is there are so many dimensions in which the software can be crap: unapproachable/overengineered, lacking in features, badly documented, inconsistent, slow, unstable, etc. The software could absolutely be some of the best in the world and still be terrible on one or more axes.

I think you're misunderstanding their claim, because you are both saying the same thing. They are claiming that juniors are unable to see (imagine, anticipate,) the cases in which _the code that they have delivered_ is crap; whereas the senior, with the benefit of painful experience, are able to predict the failure cases of this currently-apparently-working _delivered_ code.

I'm hard-pressed to see seniority being the main attribute for this rather than personality type, unless people wish to go into semantic discussions that having a more introspective and critical outlook is part of being a senior (in which case, we could easily dismiss the majority of seniors as being expert juniors).

I think it's breadth and depth of experience - both in using software, and in writing/supporting it - that is the truly correlated variable. (EDIT: though, yes, you're right that introspection and thoughtfulness also help. Someone who's been through a hundred firefights and not learned from them will not think about those things - but also, an introspective and thoughtful person who hasn't experienced or heard of certain failure cases won't have easy references or checklists to call to mind)

> I think this is one of the big differences between junior and senior software developers. You can see juniors get very frustrated with bugs because the code they delivered was 'working'. Seniors realize that software only works within well defined parameters

If that's the case, assuming those juniors have relevant degrees, they weren't paying attention in their lectures on testing. Any decent lecture series on software engineering in general, or on testing specifically, will cover that testing can almost never prove the absence of all bugs, as exhaustive testing is rarely practical. If you want proof of correctness, formal methods are the only game in town.

I like your hierarchy, that's really a neat perspective on correctness, and the tongue can be returned to its preferred posture. Perhaps a fifth entry might be useful, along the lines of As Deployed, to cover bugs in the platform (the compiler, standard library, OS, and hardware). This might not perfectly align with Working As Implemented, unless the implementation is taken to include the platform.

> If you want proof of correctness, formal methods are the only game in town.

How does this take into account unknown parameters indicated in the parent? For instance, expecting little endian but getting big endian in a different environment?

With the disclaimer that I'm not an expert on formal methods, or on endianness: endianness is something that could be formally modelled, but it seems to me the better approach is to write code that's insensitive to platform endianness. Another approach might be to ensure such platform-specific assumptions hold true by using static assertions, so that the build will fail if the assumptions are not met. A third approach might be to write (and verify) separate code for different endiannesses. Any one of these approaches should make it possible to write formally verifiable C code that behaves identically regardless of endianness.

Of course, by using a formal verification framework, you aren't worrying about being blindsided by something like endianness, you're worrying about getting the thing to verify. If it verifies, that means you haven't made any fiddly mistakes with concerns like endianness.

The SPARK language (designed for formal verification) doesn't let you write endian-sensitive code in the first place, if I understand correctly. (SPARK is technically not a fully deterministic language but it's very close. [0][1])

I don't know if there are circumstances where you must write endian-sensitive code, but don't think there are. I can imagine an endian-sensitive solution might perform better (e.g. simply reinterpreting an array of bytes as an array of 32-bit unsigned integers).

[0] https://docs.adacore.com/spark2014-docs/html/ug/en/appendix/...

[1] https://docs.adacore.com/spark2014-docs/html/ug/en/usage_sce...

> Anyone who has been in a tech support role has seen the bizarre cargo cult rituals that result from unpredictable failures.

I remember when “zapping the PRAM” was the solution for everything on classic Macs. (Maybe it still is?) It was a truly bizarre group-level tic. Literally any issue would be met with a stampede of experts bearing the magical key combo Cmd-Opt-P-R. I doubt anyone even knew what was in PRAM, but by god zapping it would fix your computer and neuter your cat.

Resetting PRAM and SMC used to be the go-to solution for everything until the security chips appeared in Macs, because nobody could figure out how to do the equivalent on those.

See the Penrose Triangle illusion[1]; what if the operant conditioning makes people who see the gap in the triangle joint find some workaround to jump over it, and maybe that training in some piece of software aligns people to see the whole triangle and not see the gap at all. Is there any research into whether people trained on some software consider it more reliable than people thrown into it?

In the 2018 thread someone writes[2] "but if I sit down with a new user for an hour, they discover all the bugs I knew about and a few I didn’t! But an hour of someone’s time is hard to get"; but the internet is full of people clamouring to give feedback about your code, desperate for someone - anyone - to listen. An hour of someone's time is hard to get, are you kidding me? I've spent hours challenging myself to find hundreds of nitpicks in something, for free, just because they asked. What did they do with it? Ignore it. More zeitgeisty look at the "MS Teams is slow" complaints, thousands of upvotes on UserVoice, one reply "we're working on it". Stuck that way for years.

It's not hard to get feedback, you don't want feedback because you don't have time or interest or spare effort or motivation to fix all the bugs, you have more fun things to do - more interesting, higher priority, more defensible. It's way better to have forum users pass around the Chinese Whispers of their grandfather's command line which he found on a BugZilla comment on an unrelated issue, which came from a mailing list, which came from an unsourced email from an internal chatroom where they found it in a fossil,,, than it is to fix that edge case. Users telling each other to clear the PRAM and run "sfc /scannow" and "chmod 777 -R" and "setenforce 0" costs you nothing.

[1] https://cdn.dribbble.com/users/6205/screenshots/5556008/penr...

[2] https://news.ycombinator.com/item?id=16866679

I referenced this article in a blog post last year. It's not very hard to tickle obvious bugs in bash's completion system.

However even though they are things you would do pretty frequently like

    echo $(basename <TAB>

    echo `readlink <TAB>

I somehow wasn't aware of them. I suspect that's because I have been using bash for something like 15 years and was TRAINED not to type those things!

Oil Uses Its Parser For History And Completion

http://www.oilshell.org/blog/2020/01/history-and-completion....

The first thing I do on a new system is add:

  shopt -u progcomp

To the bashrc. I find the dumb "just complete a f@#$$# filename" completion to be at least predictable, even if it means i need to memorize a few more arguments to some programs. Just today, I created a test-user for something and hit <TAB> in bash expecting the dumb filename completion and instead it hung for about 90s then printed a python backtrace.

So for me both of those work as expected if I'm on my bashrc...

If curious see also

2018 https://news.ycombinator.com/item?id=16863233

2015 https://news.ycombinator.com/item?id=9822637

Apparently not discussed at the time

Apple once had a requirement that all software must survive 24 hours on "The Monkey", a test setup which generated random keystrokes and mouse clicks. Fuzz testing, the early years.

Bring that back for web sites, somebody.

This is how I use all things, to be honest. It’s also worthwhile using this to evaluate statements like “Linux has great Wi-Fi, I don’t, know what you’re talking about” or “code signing is easy, you just read the error and it tells you the thing you needed to do”. Once you’ve become invested a thing, it can be hard to verbalize all the issues you ran into before and unknowingly work around, but when you use something new of course it’s “completely broken”.

Reply on news.ycombinator.com