The economics of software correctness
drmaciver.com
drmaciver.com
Monolith development is still much faster than microservice development even though microservice development is the better long term option by far.
It's all tradeoffs. I'm an architecture nut and for years I really wanted nothing more than to design the ultimate perfectly scalable and secure system but unless you're at a virtually competition free enterprise, like a telecom or an insurance company the time or budget to do so probably doesn't exist.
I've gone from "get it done" to "do it perfect" back to "get it done and avoid obvious problem starters". The reality is just that "get it done" wins the business case almost every single time.
Things like error detection and reporting in the system absolutely have to be useful. I've seen systems where that isn't the case, and it destroys productivity.
So I agree in general, but I think there are a few areas that you need to get right or you can't gitrdone effectively for very long.
People often underestimate the difficulties of getting a hack to work and overestimate how long doing something correctly will take. Even with short time horizons.
Yes. And sometimes one of those people is your boss.
Surely, there are bosses who won't listen. But of the ones I've worked for, nearly all would listen to a reasonable explanation of how best to achieve the goals, and would approve recommendations from below.
If you've got a track record of being right, that helps a lot :-)
I honestly believe the key to software development is good decision making. If you're thinking in terms of right and wrong, correct and incorrect, then you're not making decisions, you're simply doing the right thing over and over again. Only it may not be the OPTIMAL thing, or the smart thing, in any specific instance.
For example, who here really believes proponents of unit tests have never painted themselves into a corner? How did that happen? By making poor decisions.
Stop thinking in terms of right and wrong. Seriously. Sometimes unit tests are a good decision, sometimes they're not. Sometimes that hack is a good decision, and sometimes it isn't.
You make a decision on a case by case basis, not because it's right.
I agree, and the purpose of my comment was to point out that I have often observed people making decisions on a case by case basis, and repeatedly making bad decisions by underestimating how much effort the solution thought to be 'quicker' or 'easier' or 'minimal change' option is compared to the solution thought to be more 'radical' or 'unnecessary effort'.
You seem to have some specific bone to pick around testing. I'm not really interested in that. When I say 'right' or 'correct' in this context, it's generally about not being blinkered by the current state of your codebase. People get so used to the fact that their current codebase does things in a particular way, they often look at problems as situations where they have no choice but to force the problem into the frame of their code. Solutions where you solve the problem with something closer to what you would want to write if you didn't have to worry quite so much about the peculiarities of your current codebase generally scare people, but are often the better decision not just from some academic point of view, but literally from an execution, getting-things-done quickly point of view. 'correct' as I use it in this context is mainly a shorthand for that concept.
The whole point here is not to say that there's some secret that avoids you from having to make decisions, it's to say that many people in the real world are making a particular class of error in decision making. Signs that a particular decision might fall in that category include:
1. the main trade off being considered between the options is developer time
2. none of the developers actually think the believed-to-be-quicker-to-implement option is a good option apart from its supposed quickness to implement.
3. other options exist where a significant fraction of developers agree that apart from their supposed slowness to implement, they are good options.
That's something I've been saying for a while. It's all about the economics of the situation, rather than the impossibility if it.
Naturally, there are also improvements here and there that decrease bugs without increasing efforts, and those are worth looking for.
Did you mean this? Or NASA's process? Because I wouldn't characterize any of the above as "good ideas about how to [build correct software]".
Reviewing some of your other comments, I think we fundamentally disagree about how close we are to optimal software development practices. Discussing this in economic terms is like discussing the economic reasons that Columbus didn't go to the moon.
Of course, building (and verifying) CompCert took something like 10 years, so it's certainly not a feasible way to write software yet. Maybe a better analogy would be around the invention of the first airplane---we have a long way to go before everyone's flying in a jumbo jet :)
Note that Compcert has had bugs found in it too. Compcert has a verified core, but it's not a fully verified piece of software.
It's still a lot closer to bug free than almost any other software out there.
If you really want to work on super high quality stuff, there are fields where that is valuable.
C is awful. Linux is awful. Docker "containers" are awful (i.e. fancy branding around cgroups; no actual 'container' at all unlike solaris zones which did everything properly 10 years ago plus Crossbow networking 5 years ago).
We keep using (and inventing) quick hacks promoted by braggadocious personalities instead of stable infrastructure or proper tooling.
Other pieces of the economic puzzle are things like network effects and lock in. I highly recommend this book: http://amzn.to/1FTa7ib
I've released a couple of things that were tested to exhaustion, so far as anyone knows. One advantage of the event-driven approach is that you can do that.
Ok, you don't want to have a bunch of total bozo developers, or the time to get anything useful done will stretch out to infinity, but still, it's not about just 'being good'.
It's about why Microsoft with IE6 won the browser war against Netscape who made the single worst strategic mistake a software company can make by rewriting Netscape 6.0 throwing out all the code from Netscape 4.
Netscape was working with extremely buggy and convoluted code in the older version and trying to save the development community from the nightmare that is IE6, ended being late to the market with a superior product. Joel makes a very good observation that often people want to throw out old code because they think it's a mess, but the truth is counter intuitive that the old code contains vast amounts of knowledge.
A company can be first, best, or cheat and in this case while Netscape was trying to be best, Microsoft was first.
This is the reason iterative code development is best. Speed of iteration beats quality of iteration 9 out of 10 times. Boyd's Law of Iteration (http://blog.codinghorror.com/boyds-law-of-iteration/) The best software is software that released most often, not released the most correct.
If I was to release a browser in say 2008 to compete with the dominance of IE, what is the single most important feature I could put into that browser? I'd put a feature for forcing iteration, so that the browser can automatically update on the client finishing a development cycle rather than release the update preinstalled unable to remove on newly bought computers.
http://blogs.msdn.com/b/rick_schaut/archive/2004/02/26/80193...
https://en.wikipedia.org/wiki/Hyperbolic_discounting
Everyone knows that bugs are problematic eventually, it just seems that they can't put that on a level playing field with the up front costs, be they in terms of time, features, or effectiveness.
As an example, if you asked Home Depot whether they were saving money with their self checkout machines, I'm sure the answer would be different before their data breach vs after. Even after being warned they simply couldn't properly discount the possibility of massive damage in the future when offered a short term benefit.
A company that comes to market with a product that is useful but buggy will attract the attention of venture capitalists & other investors. It will receive user feedback from its existing user base. It will find it easier to hire top talent. It will be able to use collected data to make better products. All of these factors are in proportion to the company's size, which tends to make growth rates exponential.
It's pretty standard practice in the tech industry to bring a buggy, barely-working product to market; use interest in that to raise money; use money to hire engineers; and use the engineers to fix the bugs. You could even look at this as a net benefit to society, as long as existing customers would rather use the product in its buggy, incomplete state than go without it.
Also, hyperbolic discounting is explicitly not exponential in the way you might account for the future availability of resources (even at a very high exponential rate), it's valuing things on a different curve in the future than the present: would you rather have five dollars today or ten dollars next month, vs would you rather have five dollars a year from now or ten dollars a year and a month from now. People will say five dollars today, but ten dollars a year and a month from now, even though under rational analysis they should come out exactly the same.
So true.
When we were hiring for a senior engineer position a few months ago, one applicant said he wrote "bug free code."
I laughed, sent it around on slack. We all chuckled at it. The applicant did not get a call.
I'm going to come out and suggest that even THESE are more expensive than most businesses need. I believe code review processes are insanely expensive for what they usually return (wrt bug costs).
Often the time saved using a library is worth the small risk, but if we're talking about "how to write correct software" you'll want to be weary of code reuse.
Using libraries is still generally a good idea, but its effect on software correctness is a bit of a coin toss.
Another problem with libraries in the wild is that as they add features, they add bugs. If they don't add features, they lose popularity and don't get used. If they are commercial, they are less popular, and get fewer bug reports (and fewer eyeballs checking for closed source). People don't want to pay for correctness.
I think you're right: correctness is pretty far down the priority list. Good enough is good enough.
BTW: static types have correctness benefits, but dynamically typed languages are very popular - and when static types are used, it's for performance and documentation. Languages using static types for correctness (e.g. ML family and haskell) are not mainstream.
Then, your favorite language/framework will not help you much. You'll need strong verification tools, and a language and frameworks that are compatible with it.
Reality is that within a group of 10, you might as well get 12 or more prescriptions of how to write quality software. About half of those prescriptions will be useless and at least one will be actively harmful, but the team will have a hard time to reach a consensus and tell which is which.
Actually, the people most likely to tell which is the obviously harmful prescription learn pretty fast to keep their mouths shut, because they are more likely to alienate their peers than to convince them. So, everybody knows that thousand-lines-long functions are bad, but nobody can tell exactly why besides the bland, non-threatening "style" argument.