My philosophy is a little different. It grew out of writing a thesis on software quality in a post grad course. We studied lots and lots of ways to improve software quality. I can't count the number of papers I read on the subject. The first paper you read is all gung-ho about their particular method of course so you think you are done - you've found the holly grail. Then you find the second paper is equally gung-ho about an entirely different method. Eventually you become jaded by what is really opinions on the effectiveness of the technique their paper explores (dare I say delivered with the almost religious zeal of a true believer) disguised by blizzard of equations, tables and lemma's that take hours to decipher.
Being a post grad thing I had been writing software for many years before reading these bloody things. I did learn a whole pile of new techniques (my thesis got a high distinction with the comment "you sure know a lot about improving software quality"), but for most of it I was none the wiser on what I should apply to my everyday work, and that was the point of doing the post grad in the first place. So I changed tack, and started reading papers that reports the results in productivity. It's really hard to measure of course, and these papers were very thin on the ground and their figures involved a lot of hand waving. Even so I was pretty confident in the end the _only_ technique that had a significant effect on productivity was unit testing.
Note that considering quality and productivity together is very different from looking at quality alone. The most effective way of reducing defects per line is the method is probably the one developed by IBM when developing code for NASA's early space program. It relies on heavily on formal inspection. In that system programmers weren't allowed to compile their own code - and compile errors where counted in their defect rates. It was spectacularly effective at developing software with low defect rates. Programmers don't usually think of their profession of being shining example of the quality engineers of all types can achieve when they put there minds to it - but this code was just that. 100's of thousands of lines of real time code, not a single failure in flight. Contrast that to the efforts from other engineering disciplines like the O rings, tiles falling off, yada, yada.
But most of us don't develop code that must work on first deployment, with catastrophic life and death billion dollar consequences if it doesn't. In the trenches where most of us spend out working lives unit testing delivers more bang per buck then any other sort of quality measure. The reason is simple - it automates testing that would otherwise have to be done manually. Automates it so effectively that it's almost free. If you are wondering how that could have effect net productivity just read the story - the entire team stops work for days to rescue a burning site, people too scared to deploy code. But that's probably dwarfed by the lost sales while the site was down.
Still, writing tests take time. Writing too few doesn't get you the quality boost. Writing too many slams productivity. How many tests to write is a puzzle. Your post about all the tests you could write to test everything is certainly pertinent - but if you go down that rabbit hole too far you will spend all your time writing tests.
My way of deciding that point isn't like yours. In the end I decided the most important thing an easy way to add test for every line of code. The only proof I can believe is a coverage tool reporting 100% code coverage (or better yet 100% branch coverage if your tool supports it). Handing waving to explain the rest could be right, but to verify it requires a 2nd person to read code _every time it is modified_. Remember we are on about maximising productivity here, so creating manual work like this for every release is really hard to justify. Thus the first step is to write tests for your code until you have 100% code coverage.
But it goes further - it must be really easy to find what test covers a particular line of code. This usually means the same line will be tested multiple times - once by the test that targets it, and many other times by tests that happen to use it. Someone programmers view this is inefficient, and it is in a way, but you will see later there is a reason to spend this extra effort. Sadly it is time consuming to verify this has been done in every case as you would have to check every line of the code being tested, but it is amenable to auditing: if you select 10 random lines and they are all good the odds another random line doesn't have an easy to find test is 0.1024%.
As anyone who has written a lot of tests knows, this insistence on 100% test coverage has flow on effects to the code being tested. Arranging code so it is easy to spot how you get to the line you want to test can be the most tedious step. It's not unusual for the "writing tests" step to force the code being tested to be refactored for visibility. It is this "refactor your code so it is easy for others that code later to add more tests" that is real goal of the 100% test coverage condition.
Once they've hit 100% then, and only then, can the programmer take into consideration the sorts of things you describe. It's difficult to check if they have done that of course - at doing such checks really requires 2 people to think up the tests required, which doubles the work. But perhaps it doesn't matter as at lot will happen anyway if your programmers are diligent in developing test for every subsequent bug you hit. Here diligent means thinking about all the other places this bug may arise, and testing it there too. Relying on this is effectively leveraging off the all the systems testing that follows to produce the sort of tests you describe automagically.
Adding these new tests is easy of course because you already have 100% test coverage, and its obvious what code tests each line. You can ensure adding these new tests happens by insisting every programmer report every bug they find in their own library or someone else's into a bug database, and later record the test.
Doing it this way does not guarantee your code will have a 0% defect rate. But then nothing, not even what NASA did can do that. What it does guarantee is over time your defect rate will drop, and it will do that with minimal extra work. I find the "extra work" bit is important, because when you are in the "creating the new shiny" mode finding the energy and money to do the work isn't so hard. Hopes should be high is the next big thing that hits the jackpot. But once you are in maintenance mode reality has hit home. Every new addition has to justify every cent spent on it. I suspect the that is because existing KPI's are in place, everybody has a reasonable feel for the impact a change might make - and most changes are small increments. Yet over the years most development happens via the cumulative weight of these small increments. Ensuring the cost of maintaining 100% test coverage during these small incremental changes is minimal is only hope you have of ensuring it's done.