Understanding the rationale behind a rule when trying to circumvent it | Hacker News Reader

Understanding the rationale behind a rule when trying to circumvent it

devblogs.microsoft.com

130 pointstosh36 comments

Am reminded reading this of an esteemed and since passed away colleague who had written windows driver code since the dos days and may have had decades of insanely archaic knowledge die with him - when working on a difficult piece of windows driver code years ago, he said to me in a thick eastern europe accent as best i can remember “you make the primary mistake of thinking anything in windows makes sense. once you abandon this bias, you may someday hope to get where i am”

We all knew this kind of guys. Bless them

If there's anything I've learned from reading Raymond's blogs, it's that Windows internals actually make a fair bit of sense. Every weird thing I've ever seen him address turns out to be something that has had real thought and engineering put into it, and if the solution is ugly that seems to be more due to external constraints placed upon the engineers than it is the engineers doing weird stuff shooting from the hip.

Implemented by intention vs implemented by history

This seems adjacent to Chesterton's Fence, though maybe not the canonical form of it.

For anyone not familiar with the term, Chesterton's Fence is the idea that you should understand why a rule exists before trying to remove it or work around it: https://fs.blog/chestertons-fence/

Here the issue is not that the rule was removed, but that the code followed the wording while missing the reason the rule existed.

Came here to say this. How can you write an entire article about Chesterton’s Fence without mentioning Chesterton’s Fence?!

Chesterton's fence has two parts:

1. I want to remove a rule

2. Understand why that rule is in place before proceeding

This article deals with the second part, but not the first. So it is only about about half of Chesterton's fence at best.

In these examples, a rule (avoid blocking calls) is in place to guide the programmer to a performant system. Programmers apparently thought that if they found a way to avoid directly blocking calls, but managed to indirectly block, they had still obeyed the rule. And strictly by the most narrow reading of the rule they had obeyed it. But they had defeated the purpose of the rule.

So definitely Chesterton's Fence adjacent, but not Chesterton's Fence itself.

It's not about Chesterton's Fence, though I thought so at first due to the title wording. The developers Raymond is referring to actually weren't trying to circumvent a rule. They followed the rules to the letter, but whoever made the rule didn't adequately document the spirit of the rule. So when the letter of the rule turned out to be insufficient, the devs couldn't really know that their solution violated the intent behind the rule.

So we can call this one Chen's Fence

Yeah, I agree. Often prohibitions are made without any context as to why they exist.

I remember back in the 80s when I was a young kid (maybe 8 or 9) and I'd just started learning Z80 machine code. The manual for the computer had a strongly worded "do not use the alternate register set" (probably bolded and all caps), and sure enough the one time I tried, the computer crashed. I decided that it must be dangerous, and never tried that again.

It was only years later when I discovered that the OS firmware used the alternate registers in the interrupt routines, and assumed that they were always set a certain way (in particular, BC was used to ensure ROM was visible not RAM before jumping into it, and on return RAM was paged back based on the existing contents of that register). So, if the warning had said "don't use the alternate register set unless you disable interrupts and save/restore the registers before re-enabling them", it'd have indicated what the problem was and why. Additionally, I think somewhere else there was a dire warning about "disabling interrupts for too long" with no indication what was too long or what problems it might cause. In reality the only thing it affected was the built in system clock which was only used by BASIC and the sound firmware, so for almost everything you might want to do it didn't matter in the slightest. (Actually, there was a slight caveat with the hardware that you actually did need to acknowledge the vertical sync interrupt within a certain time frame or it would fire at the wrong time on the next frame, but even if you did that, it would correct itself by the second frame, so again was entirely unimportant, especially if you never planned to return to BASIC).

As well as not explaining the why, which helps understanding of the system generally, a list of prohibitions makes experimenting with the computer seem scary. One of the best things about old 8-bit home computers was that there was very little you could accidentally do in software that'd actually cause any lasting damage, unless you were deliberately doing stuff like toggling relays as fast as possible or fiddling with the monitor sync signals, sending excessively out-of-range timings and then left the monitor for a long time.

Maybe because every concept has some plonkers name associated with it. The ideas stand on their own. It does not matter who said what, when. I'm pretty sure the only reason people feel the urge to interrupt a conversation to drop names like that is to boast about their useless knowledge.

Agreed that on first blush it looks like Chesterton's Fence but it may be more of a cargo culting situation.

There is this famous experiment with 9 monkeys in a room with a banana attached to the ceiling and a scale.

First day, a monkey climbs the scale, gets the banana and is happy.

Second day, they start spraying whomever gets on the scale. Monkeys hate this. They learn not to climb.

Third day, they take a monkey out and replace with another. The new monkey sees a banana up there and tries climbing the scale. He literally gets beaten out by the others, like "seems like you're new here".

Days 4-12, they've replaced one monkey per day, so that no monkey was here when it was possible to get the banana. None of them have ever been sprayed either. Still, they enforce the rule not to climb up there.

I am putting this example because in our society as well, there are many rules that are enforced without anyone questioning the "why". Yet the "why" is often more important to know than the rule itself.

Designers know this dichotomy between the "why" and the "how". Most people don't.

I've heard that story. I've yet to see evidence it actually happened though. I don't the experiment with pass a modern ethics panel either.

parables that didn't happen can still be useful

They can be, but you need to take care as sometimes they are misleading and can even drive wrong decisions.

Aesop wrote a lot of fables that had contradictory lessons.

There's a meta-lesson there.

I have more than once in life realized that one of the two contradictory lessons was correct for this situation, but in a different one the other was correct...

The parable-replication crisis is real though

I stand corrected: https://blogs.cornell.edu/info2040/2012/11/03/monkeys-and-in...

It's funny how this monkey experiment is often trotted out as "people blindly follow the rules without knowing why," when the rule learned by the monkeys is a really good rule (it prevents a malevolent entity beyond your understanding from attacking you!), and the only reason the monkeys don't understand the "why" at the end of the experiment is because they don't have language.

The lessons I'd take away from the experiment would be 1) be sure to tell people why the rules exist, but also 2) follow the rules even if there's no apparent reason for them, otherwise you might get smacked down by some unimaginably powerful entity you're barely even aware of.

I feel like a lot of documentation falls into this kind of style, where the motivation behind design is not communicated, just the design itself. Sometimes it's because the documentation writer doesn't want to leak internal details to the end user (closed source libraries are an especially bad source of this). Sometimes it's that the writer is too close to the project, and is struck by the curse of knowledge (can't properly identify which details are actually self-evident, and which are only obvious because they already know them).

Another example of why technical writing is difficult, I think.

I worked on a team where the responsibility of maintaining the onboarding guide was always passed to whoever joined right after they finished going through it (with the current maintainer being the "onboarding buddy" of the new team member and passing the torch at the end). The rationale was that the longest-tenured members of the team would be the most likely to accidentally rely on implicit knowledge when updating it and forget to put in something that wouldn't be obvious to a newcomer, and the person who had needed to learn the processes from scratch most recently would be the most likely to notice gaps like that (and could reach out to the rest of the team to have the gaps explained to them so they could fill them in).

I always liked that mindset, and it helped teach me the important lesson that sometimes being the person who asks very basic questions because I don't understand things can make me valuable to the teams I work on. There are certainly times when the answer makes me go "oh, duh!", and I feel a little sheepish, but the feeling doesn't last very long, and no one has ever held it against me, whereas the times when it leads somewhere interesting and potentially to better documentation for people coming after me are far more frequent and memorable to everyone involved.

This isn't to say that the burden should fall on the ones who are reading the documentation rather than writing it, but to encourage people who are in the position of being frustrated and confused to take advantage of those moments, because they can make you very valuable to your team in addition to helping you learn. If you're on a team that operates in good faith, the burden for documenting things well can be shared, and in the long run it will matter less who's job it is to keep it updated and more about whether everyone is contributing however they can to maintain the quality (and if you're on a team that operates in bad faith, you have my permission to keep quiet and do whatever you can to get through that experience, not that you need it from me!)

I really wish Windows had a way to show what kernel callbacks are registered and execute on each call. Even better would be to have actual timing data on these callback. Using that one could much more easily to debug and potentially uninstall misbehaving software.

You may already know about it, but for those that aren't aware of it, Windows' Event Tracing system can give a pretty deep view into identifying misbehaving software. It probably doesn't cover everything you wish you could track, and not quite with the level of directness you're asking for, but it's helped me track down weird slowdowns and hangs on my computer on a few different occasions.

There's a third party tool for collecting and examining such traces easier called UIforETW

https://randomascii.wordpress.com/2015/09/24/etw-central/

How many of these winapi callbacks should have been just messages?

Microsoft is so broken that an employee finds it easier to write a blog post about a documentation improvement than simply making that improvement? Explains a lot. "Conway's Flaw?"

the blog says the documentation improvement was made in 2020? presumably not by the post author either

From the article:

> The documentation should open with something like this:

>> The callback function must perform its work quickly without blocking. If you need to do complex work or synchronize with other threads or processes, do the work asynchronously, such as by using System Worker Threads.

A change was made, but not the change that Raymond thinks would explain why the list is there anyway.

Always been that way

Raymond's important and high up enough that he probably only needs the approval of three AIs and five managers to publish a blog - updating documentation likely needs twice that.

He used his blog to queue the work to be executed asynchronously by another MS worker that reads it

And, since he isn't waiting on the reply, he is following the rule for the callback.

of course they had to add dont wait for your worker thread in 2020

Reply on news.ycombinator.com