Criminal Overengineering
coderoom.wordpress.com
coderoom.wordpress.com
Apparently there are two very popular types of article in the software blog world:
Type 1: YAGNI (like this example): Do less now. Refactor later as needed. It won't be needed, most likely. Chill out. (All driven by the question, "Dude, wtf? 100 lines of boilerplate for a 5 line case statement? Snap out of it.")
Type 2: Architect astronautics: Do more now. Build for the next version. You will need more then, so why not prepare now? Decouple that code. Use more patterns. Hoist that jib. (All driven by the question, "What will your code/software/app do if...?")
I read type 1, and it (often) sounds convincing. I read type 2, and it (often) sounds convincing. I get a fucking headache from the cognitive dissonance. I make more coffee and get back to work, no wiser than before.
1. You can read anything and it (often) sounds convincing. Quality writing that is incorrect often trumps poor writing that is correct. That's way we must always be vigilant readers, especially on the internet.
2. "ArchitectingForTheFuture" does not necessarily mean "MoreLinesOfCode". I'd like to read more articles of Type 3: How excellent design and proper use of tools gives you the best of both Type 1 and Type2.
[EDIT 1: stcredzero, in reference to cousin comment, my personal metric is n=2. I never want the same line of code more than once. I'm sure there are good arguments for other values of n, but this has always seemed to work well for me.]
[EDIT 2: I try to make a point to never say "Not Hacker News" or complain about content. Conversely I should be quick to claim "Not Not Hacker News". This is a great thread! About stuff near and dear to this programmer's heart. Keep 'em coming.]
The advantage of this approach: you never have to prognosticate. Hindsight is 20/20, so use it!
EDIT: I used to use n=2 as my threshold, but I found it much better to have a slightly higher n in Smalltalk. I am spoiled, though because I have such lightweight but powerful (syntax-aware) tools for searching such patterns.
And so, it becomes a cost-benefit analysis. C(YAGNI) = C(develop a simpler version) + C(rework) x p; C(AFTF) = C(develop) + C(maintain). Where p is the anticipated probability that you will actually need that feature (and realizing that you're probably going to guess higher than the actual probability).
Isn't that why we write good unit tests and refactor? It gives you close to best of both worlds. Writing unit tests is helpful anyways, and gives you peace of mind when you introduce patterns later on when you actually need them.
I think you're misinterpreting Type 2. You don't bring out Type 2 to prevent future problems. You bring it out to solve problems.
I have a rule: don't apply a pattern or other advanced technique until doing so will eliminate or reduce code in more than 3 places. If the same code starts popping up in more than 3 places, it may show up in a whole lot more, and it's going to start being a pain to find all of the occurrences. On the other hand, just finding 2 or 3 and correcting them is not really that hard, so leaving things until n=4 is not too bad.
Going back to the article, never apply a pattern in a language until doing so yields fewer lines of code than the original code. If your language requires N > 10 for this to be true for most patterns, then switch languages! (Someone should turn this into a metric!)
An ideal of clean, logical design actually satisfies both #1 and #2. You don't write any more than you need to, but you architect it sensibly so if you need to modify it in the future, there's a natural way to do it.
A case study: you are responsible for maintaining a moderately sized mailing list.
Type 1: Maintains a comma+newline separated list of name/email pairs in a text file, parse it with a 3 line perl script that sends an email for each regex match. Done in 15 minutes. YAGN anything else.
Type 2: Builds a fully relational model in a database with seperate tables for names and addresses, backed by Hibernate with a complete class hierarchy including AbstractRecords (in case you want to store records other than name/email pairs), RecordFactories (In case we need to generate lists of records from another source), AbstractRecordDAO (in case we need to use a different Database or ORM framework), EmailFactories, AbstractMailerImpl, etc, etc, etc, planning in the architecture for anything anyone might ever want to do with a mailing list.
Both of these are wrong, IMO.
The correct solution is to write one database table, or one cleanly formatted file, with a program with perhaps two classes that abstract apart the data loading and the mailing tasks. One class loads the data into a simple Map structure, and the other that handles iterating the map and doing the mailing.
It only takes a bit longer to write than #1, and is infinitely simpler than #2. It doesn't anticipate every future need, but when one comes, there's a logical point to start adding the functionality. If I have to do something new with the list, I don't have to completely rewrite my program (like #1 does), I just write a new class that uses the existing data structure. Just as extensible as #2, with a tenth of the work. And easy to see what's going on.
If I have to do something new with the list, I don't have to completely rewrite my program (like #1 does)
You said #1 only took 15 minutes. Hardly seems like a big cost. If you never have to rewrite, #1's the right answer. Or if the rewrite includes functionality never even anticipated in your "correct solution". If it's that small, starting over with a clean slate may be faster than fitting in new functionality.
But say instead that it's a larger problem, and that #1 takes one day to build, #2 takes a week, and the "preferred" solution (call it #3) takes two days.
Then along comes your change request. It will take most of a day to rewrite #1 to do something never anticipated, but it can be slotted into #2 or #3 in about an hour.
Now you've broke even between #1 and #3, and that's only the first change request. Any real system is going to get many more.
These numbers are pulled out of thin air, of course, but they're pretty accurate according to my development experience.
After 15 years of writing code I can only vouch for two metrics of code quality: clarity and length. For a long time I too thought that clarity means using "thisIsAStudentBirthDate" variable names - but it turns out there are very very few instances where the shortest code isn't also the most obvious at first glance.
I think the turning point was when I decided rewrites aren't a bad thing. I started seeing the code as alive - unchanged code isn't good, is dead. The myth of perfect code is just an illusion. Writing for the next version sounds very sensible - until you get enough years under your belt to realize that either there won't be a next version or it won't be anything like you expect.
And rewrites do become difficult as a system grows if you don't think them through. It's easy to rewrite one component with no dependencies, but it's not so easy to rewrite something that lots of different pieces of code rely on.
Like I said, what eventually bites you is that you never ever ever know exactly what version 2.0 will require. So in the end you still have to adapt code, and adapting clean short code is easier.
When in doubt, remember what Knuth said: "Premature optimization is the root of all evil."
I had an argument with a teacher that thought that it would be useful to abstract things from a pop/imap email fetching service, in case other protocols show up ? I'd argue that was improbable and that would complexity the design and that possible other protocols might be so different that they wouldn't fit the abstraction. He dismiss it as industry specific way of doing (but I lacked examples a little bit, so I wasn't completely convincing ).
We have to be quite vocal because we cry against the mountain.
> If you're about to take a hundred lines to write what you could in ten, stop and ask yourself this: what the fuck?
TDD advocacy is my pet hate for this today. I read an article the other day that managed to turn Hello, world into about half a dozen source files and dozens of lines of code, all pulled together with a makefile several times longer than the whole effort should have been, and seemed to be claiming some sort of profound revelation from doing this!
The quote is also a succint analysis of why C++ and Java are becoming less useful every year, relative to the field of programming languages as a whole.
And of course, as in the example in the article, it's good for bashing people who think tools like design patterns must be used everywhere, regardless of whether they actually help to keep a design clean and maintainable.
http://blog.objectmentor.com/articles/2010/05/20/hello-world...
I don't often visit their blogs any more, but occasionally, I still come across their site after someone links to it. It's one of those things where you know it's bad and you should just look away, but somehow you can't help yourself. ;-)
1. Is verbose by its very nature 2. Has language characteristics that push the developer in the direction of using design patterns at every opportunity.
So what might be a 1000 line program in Ruby is a 3000 line program in Java (translated as directly as possible), but then the deficiencies of the language turns simple idioms in a powerful language into complex design patterns in Java. See: closures, lack of duck typing/deficient type system, etc.
It would be easily possible in other languages to write 4 implementations of a basic list or public setter/getter for every member or buffered stream interfaces for all kinds of I/O.
Its not done because long solutions are frowned upon.
KISS: Keep It Simple and Succinct
Succinct means brief and concise, to the point. A succinct argument is one that more directly addresses the point under discussion.
The "traditional" meaning of KISS completely misses the point. The biggest problem KISS addresses is over-complication of plans - and it is not a problem of stupidity. Those most prone to over-complicate are the more intelligent, especially the highly intelligent and highly educated, but lacking in practical experience. Experience, especially wide experience, is the best prophylaxis for over-elaborate plans.
http://williambswift.blogspot.com/2009/04/kiss-keep-it-simpl...
1 - only solve problems that already exist
2 - only accept solutions less painful than the problem
That sounds good, but it doesn't work unless you have the experience to tell how painful the solutions you thought of actually are--which you don't have if you haven't been hurt by some overengineering.
On the general principle I agree that overengineering is to be avoided, but I actually think the example shows a clear disregard for Object Orientated principles.
Contrary to popular opinion using OOP does NOT mean "thou shalt make every last thing an object"
It's not called Object Programming, but Object-Oriented Programming.
If we all called it "Class Oriented Programming" we'd come closer to an accurate name. "Class Oriented Programming" as a name might take away the emphasis on instances, and put emphasis on designing classes.
Or not. There appears to be no bottom to human stupidity.
I suspect that in Self, it's easier to make more things objects. (jk - everything is an object in Self.) Objects aren't quite as easy to use in C++ and Java. The cost is higher, so the opportunities to use objects with a good cost/benefit payoff are fewer. That's all there is to it.
Does this generalize? In most Functional languages, functions are really easy to use, and can be used in flexible and powerful ways. What's the best way to program in them? Why, using functions! Yup, seems to work. Fancy that!
But not everything must be abstracted :)
Even if more operations had to be added I'd probably let it grow to the point where the method the switch is in was getting a bit unwieldy then look at refactoring it using a pattern if I really thought it would be worth it.
Because you won't always have only three operations. What about division? Exponentiation? Square root? Factorial? Arbitrary user-defined functions? What if you didn't anticipate an operation one of your clients needs? If you use standard OO principles, your client can rectify that problem; if you use a switch statement, they can't.
> Even if more operations had to be added I'd probably let it grow to the point where the method the switch is in was getting a bit unwieldy then look at refactoring it using a pattern if I really thought it would be worth it.
Why not just do it right from the start? It's extremely simple, it's a pattern every OO programmer is familiar with, it's more computationally efficient and has other advantages as well.
It's also not nearly as complicated as the linked Strategy code. The appropriate analogue to the author's switch statement (in Python, since I'm not a Java programmer):
class Op(object):
def eval(self, a, b):
raise NotImplementedError
class Add(Op):
def eval(self, a, b):
return a + b
class Subtract(Op):
def eval(self, a, b):
return a - b
class Multiply(Op):
def eval(self, a, b):
return a * b
This design (it's the basic "convert a switch statement to polymorphism" refactoring) turns a ball of mud switch statement into several independently comprehensible classes. Operations can be understood, verified and tested apart from the whole evaluation apparatus.
No doubt many programmers today massively (perhaps even criminally) overengineer their software, but the fact remains, the example chosen by the author is a really bad one.
When something is not extended as expected, either you end up with something not as good as you'd have gotten by waiting or you have to rip out some of what you put in for the future without ever using it.
You know, everyone says this, and I've never quite gotten it. Why is writing another class and implementing another virtual function so much better than adding another clause to a switch statement? At least all the cases of the switch statement are in the same place instead of scattered across a bunch of files.
Polymorphism to me just seems like a switch statement you have to think harder about. Maybe that's why I've always preferred a functional style to an object-oriented one.
Disclaimer: I program in Fortran for a living, so "SELECT-CASE Stockholm Syndrome" is definitely a possibility.
1. Polymorphic method implementations are lexically isolated from one another. Variables can be added, removed, modified, and so on without any risk of impacting unrelated code in another branch of the switch statement.
2. Polymorphic method implementations are guaranteed to return to the correct place, assuming they terminate. Switch statements in a fall through language like C/C++/Java require an error-prone "break" statement to ensure that they return to the statement after the switch rather than the next case block.
3. The existence of a polymorphic method implementation can be enforced by the compiler, which will refuse to compile the program if a polymorphic method implementation is missing. Switch statements provide no such exhaustiveness checking.
4. Polymorphic method dispatching is extensible without access to (or recompiling of) other source code. Adding another case to a switch statement requires access to the original dispatching code, not only in one place, but in every place the relevant enum is being switched on.
5. As I mentioned in my previous post, you can test polymorphic methods independent of the switching apparatus. Most functions that switch like the example the author gave will contain other code which cannot then be separately tested; virtual method calls, on the other hand, can.
6. Polymorphic method calls guarantee constant time dispatch. No sufficiently smart compiler is necessary to convert what is naturally a linear time construct (the switch statement with fall through) into a constant time construct.
Now, to answer the objections you offered. You said, "At least all the cases of the switch statement are in the same place instead of scattered across a bunch of files", to which I would reply that in the polymorphic method case, at least all the code related to a particular case is in the same place. In the switch statement case, you're spreading dispatch machinery and data-specific code all over your program, wherever you switch on an enum. In the polymorphism case, that dispatch machinery is abstracted by the compiler (and thus not present in your code at all) and all the code related to specific types of data is centralized in that type's class. The general code remains general, having no knowledge of the specific, per-type implementation.
You also said, "Polymorphism to me just seems like a switch statement you have to think harder about" to which I reply that on the contrary, polymorphism is great in that you don't have to think about it at all. A programmer attempting to demonstrate that a switch statement is correct must delve into the switch statement and show it to be correct for each case. A programmer attempting to demonstrate that a polymorphic call is correct need only ensure that the call's abstract preconditions are satisfied, and can consider the actual implementations of that call to be black boxes that he need not look into.
In response to your disclaimer, I would say that I mean no offense, but it's very possible that Sapir-Whorf is impacting your language preferences here. You find what you do most to be easiest to understand, and what you do most is switch statements, not polymorphism. I am no doubt afflicted with the same condition, but I think as I demonstrated above, there are many objective reasons why polymorphism is superior to a switch statement in most cases.
If the other 5% of the time stuff takes off and becomes standard throughout the projects using your code, then it can more than make up for it.