Frequently Forgotten Fundamental Facts about Software Engineering (2001) [pdf]
kictanet.or.ke
kictanet.or.ke
The "28x productivity" (or 10x or Xx ...) claim always triggers warning bells for me. http://morendil.github.io/folklore.html does a pretty good job explaining how the research backing that widely accepted "fact" may be questionable.
I also really like Dan Luu's review of the research behind static typing at http://danluu.com/empirical-pl/ as an example of how a "well studied" claim can still be questionable due to the massive difficulty in evaluating software engineering empirically.
Software engineering is a very human activity and that makes it very hard to measure and quantify.
A book that does a better job is Making Software (http://www.amazon.com/Making-Software-ebook/dp/B004D4YI6G/re...) but as my first link points out, it still has some issues. At least its goal is to get more rigorous in our scientific analysis of software engineering.
I don’t expect you to agree with all these facts; some of them might even upset you. Great! Then we can begin a dialog about which facts really are facts and which are merely figments of my vivid loyal opposition imagination!
It's interesting that he was (is?) a vocal dissenter on Open Source.
>in 2000, Glass criticized open-source software, predicting that it will not reach far, and "will be limited to one or a few cults emerging from a niche culture." Glass's basis for this bold prediction was that open-source software "goes against the grain of everything I know about the software field"
Not to judge off something that small, but if "everything he knew" told him open source was not going to reach far, then perhaps his other facts are off.
There are no references. They look more like random opinions - some interesting, but with a fair sprinkling of platitudes - supported by some equally random numbers, of the "Did you know fifty percent of statistics are made up?" kind.
I think it's interesting how dated the piece looks. Equivalent writing today on Quora, Medium, or HN - never mind an IEEE journal - would be more likely to discuss real research.
It might not be certain to have facts, but I think standards of argument have improved significantly - possibly because it's so much easier to find and reference studies than it was when a lot of debate happened very slowly in print.
We have the notion of cyclomatic complexity, an empirical measurement that can be used to assess the complexity of a piece of code, and presumably that code is the solution to a problem. So, OK, we have something like a measurement of solution complexity, but how is problem complexity measured, and what is its empirical relationship to the measure of complexity of the solution? Hell, what are the units we would measure such complexity, even?
I think people who do what we do have a certain insecurity about how much of what we do is decidedly not science. And we have this tendency to talk in science-y ways about what we do, which sound good to our ears, but aren't quite rooted in anything concrete.
So, when people start talking about the cyclomatic complexity, it seems to me that we are better off understanding we are deep into a philosophical discussion rather than a scientific one.
(http://www.scirp.org/journal/PaperDownload.aspx?paperID=779)
We found that due mostly to issues regarding population variance, that the linearity of the relationship between these two measurements has been severely un-derestimated. Using modern statistical tools we develop linear models that can account for the majority of CC by LOC alone. We conclude that CC has no explanatory power of its own and that LOC and CC measure the same property. We also conclude that if CC does have any validity as a measure of either complexity or test space size, then we must conclude these factors grow linearly with size regardless of software language, paradigm, or methodology.
I was looking for another reference, and I found this one, which seems far better researched.
¹ For example, here: http://blog.codinghorror.com/revisiting-the-facts-and-fallac...
Hmm, doesn't this conflict with the idea of "rapid prototyping", where new requirements are thrown in whenever necessary?
They will also avoid allocating time/resources to fix modules that are known to carry significant technical debt. Only until said technical debt begins to cause severe problems any official attempt to address the issue will be done.
In the mean time, savvy senior engineers will fight a guerrilla war to keep technical debt at bay. They will require, incremental improvements to be carried out in parallel with bug fixes and new feature development, typically whenever you touch a file/function known to be in a bad shape. This is more or less OK, but can result in an irrational fear in the team to change stuff for any non-essential reason, eventually defeating its own purpose.
At the end, I always assume that whatever code, no matter how crappy, I get to commit into the source control system, will eventually find its way to our customer's machines. No matter what claims of temporarility are made by people, if it gets out of your localhost, it will be promoted to officialdom in no time.
Then we had a calm discussion about how long it would take to productize it (usually weeks or a couple of months). It might not seem like that big of a coup but attempting to ship 2x as much as you actually can is a sure way to be absolutely miserable by version 2.1 or 2.2 of the product.
Maybe it is possible to marshal user expectations by carefully controlling how much of the GUI is produced. Not make any pages/windows until the underlying business logic is production ready... it's a wild guess, but might work
Only a half joke, too: it's easier to critique and improve something concrete, even if it's objectively rubbish, than to go from 0 to 100% from just abstract discussion.
I disagree with that conclusion. Prototypes help in both clarifying requirements as well as eliciting requirements. However, for the latter, a prototype may be necessary but not sufficient.
But instead, if there are big red Xs and so on in the UI, then it becomes clear "oh this is a prototype/demo/".
* Having small, self-contained, loosely-coupled modules.
* Having an extensive test suite for each module.
* Having small (or zero) amount of technical debt.
* Talking to your customers constantly, preferably before you commit a lot of resources into a new development.
Easier said than done, but definitely not impossible.
"COBOL is a very bad language, but all the others are so much worse."
So often I find the first step is to back up from their proposed solution, by gently probing what their reasoning is, then once the problem is better defined, only then going forward again sketching out a solution.
I always get people wanting another "email when this happens". When I dig bit more into the details, a confirm page on save is a far better solution.
I find the worst are semi technical managers who have been promoted too early, as they come up with junior level programer type solutions and think they are being helpful working out that for you.
The real savings with "agile" methodologies is the understanding that the code is the documentation. This doesn't free you from having requirements or design documents, it just allows you to spend less time on that part of the process. For any sufficiently complex project not having block diagrams of how everything fits together, and basic documentation of subsystem interfaces just means you waste a ton of time reading the detailed implementation before you can understand how the system works.
In other words its the same problem you have with heavyweight processes. If you have to read 500 pages of design documents to understand how to integrate your routine, that is the same has having to read 50,000 lines of code to understand how to integrate a piece of code.
Agile also shouldn't be an excuse not to document. My (perhaps not fully informed) view is that it's more about iteration.
Neither Agile nor Waterfall are going to save me from a pain in the arse of a change that affects a lot of the application.
Agile could have helped uncover the need earlier, or enabled the project to react to it mid-project. Post production changes don't seem better or worse served by either.
The project is an in house database tracking the samples and pretty much everything else for a DNA sequencing centre. The (sequencing) tech changes fairly rapidly and database migrations are frequent. It is constantly mid-project.
If I asked three and a half years ago and was promised, "no it won't ever happen", how would "agile" have helped uncover it earlier? Would a load of unit tests written before have made a difference? A stand up meeting every morning? A scrum master? Sorry, that's just bullshit.
It came up when it came up, a couple of weeks back. For the time being the workflow in the organisation will have to workaround. Its not a common case, and its really not worth the effort at the moment.
For the record I don't do TDD, Scrum type Agile. I do follow a lot of the principles in the agile manifesto, I have to. I do what works for me and the organisation, and blindly following methodology wouldn't.
a "flexible process" sounds great. However, the reality is that when you allow flexibility, the project rarely ever gets completed on time or within budget..which also seems to be the main requirements for a business.
I've seen it way too much.
"...except for the additional maintenance task of “understanding the existing product.” This task is the dominant maintenance activity, consuming roughly 30 percent of maintenance time."
I've definitely lost a large chunk of my programming life trying to understand unclear code in large systems!
Edit: removed ambiguous quantification
http://www.redbubble.com/people/ramiro/works/13306366-im-a-v...
)-;
All other section from the essay are interesting too, and most are still very relevant - 14 years later!
Thanks for posting this.
P2. Good programmers are up to 30 times better than mediocre programmers, according to “individual differences” research. Given that their pay is never commensurate, they are the biggest bargains in the software field.
Much of the list stands the test of time, though it's very clear that his thinking predates Agile. Most old school software engineering experts talk of the need to tighten up unstable design. Modern thinking is to create processes that better react to the instability.
The 30x ones usually have a professional reputation that preceeds them - you know before the interview starts, and spend the hour or two selling them. They don't send resumes out. You either have to build people into them, or go hunting once you hear that their companies are struggling. (Or if you hear they are being mistreated)
Of all these topics he talks about, I think this is the one that has changed the most in 14 years. Open Source librarys, github, etc, have made reuse-in-the-large so much easier and it's so much more common now.
Horizontal re-use has always been possible but vertical reuse is a pipe dream.
p.s.: we sell certifications!
(repeat for a new Silver Bullet roughly each decade)
One item on the list I believe would change: REU2, reuse-in-the-large. With so many new services available, the number of addressable "common" use cases has become very granular. So, reuse-in-the-large takes on a new definition for me.
> REU5. Pattern reuse is one solution to the problems inherent in code reuse.
Patterns are just Reuse in-the-small anyways.
Q2. Quality is not the same as satisfying users, meeting requirements, or meeting cost and schedule targets. However, all these things have an interesting relationship: User satisfaction = quality product + meets requirements + delivered when needed + appropriate cost.
As someone who specializes in quality, I'd say either he was wrong, or the practical definition has shifted.
What he calls "user satisfaction" is what I would equate with "product quality," at least in terms of what we mean when we aim for a particular quality bar before releasing. The various factors he lists as comprising quality are aspects of satisfaction that may or may not apply to a given audience of users, but the absence of one or more does not necessarily indicate low quality unless it's relevant to the user.
Compare Twitter when it started with Twitter now, for example. When it started, it was a relatively unreliable product, but still high quality for its set of users--at the very least, it was high enough quality that spending time and money to raise it may not have been a good idea. It was successful as it stood. Now the set of users and their expectations have shifted, both because the field's bar has raised in general and because it has enterprise use, so reliability is a much bigger deal. It's still high quality, but for different reasons.
Some of those things (portability, testability, modifiability) are simply conflating code quality with product quality, unless what you're developing is a code component. Even efficiency is meaningless to quality unless inefficiency causes the -customer- to bear more load. Maybe he meant to conflate those things, but I don't think they belong together. They're rather orthogonal: vim is a high product-quality app with (supposedly, haven't read it) pretty awful code quality. Conversely, I've seen plenty of pretty codebases that produced crappy apps.
And meeting requirements, delivering at an appropriate time (which is another form of meeting a requirement) and at an appropriate cost (yet another form of meeting a requirement), these are all absolutely important aspects of quality. The scale of quality a free-as-beer product is measured on will always be different than one a paid product will be measured on.
Basically, he's taking a very absolute approach to quality, rather than considering context. There's really no such thing as a "high quality product" in the absolute. It's all relative to the intended audience and use.
I will agree, though, that quality goes way beyond sheer absence of objective software defects. It's a shame that most of the industry tries to define it that way.