How to Design a System
--
The second system effect is when a designer isn't new on the job anymore (he's made at least one system) and he has an overconfidence in his abilities. Because of that, the second system he designs is usually an overcomplicated mess. There are exceptions. Woz may not have designed an overcomplicated second system, for example. And if you understand how little you understand when you're first starting out, you're more likely to build small, stable systems instead of large, brittle systems. But history has pretty much proven that inexperience + overconfidence = mess, regardless of your profession.
The first system usually goes well because you're careful. You want your first system to go well since you will be judged by it. You keep your opinions to yourself and use proven methods that you fully understand to design your system. After you've designed this beautiful little gem (or not-so-beautiful but working, so the creator may think it is beautiful), you're left with a feeling of accomplishment. In fact, that first system wasn't really hard at all! You read the requirements and delivered a system that did exactly that. But look at how little that first system accomplished. In fact, it was even rigid; it did exactly what was needed without an eye towards extensibility. You know you can do better.
But.. It seems to take a lot of experience to correctly design a large, extensible system. I really hate to say the e-word since (being 19) I hate the idea that there's this magical quality, experience, that you'll only get after a certain amount of time has passed, regardless of anything you do to try to accelerate the process. But it seems true. It seems like to be good designers, we need to immerse ourselves in as many projects as possible; understand that we know very little; and most importantly, get things delivered, even if they are only at the 80% mark.
The Mythical Man Month is a good read. Honestly, it's extremely.. well.. boring. The advice was penned for a very different time, so a lot of it doesn't apply today (a programmer needs a personal secretary? That would be nice, but probably not required). But if you read it through the shades of, "Wow, he is a really creative thinker!" then you'll enjoy it a lot. He talks about why the best designers have usually created at least two systems. The first gives them experience, and the second gives them humility.
Choose two: beautiful, meets all requirements, stable.
"Beautiful, and meets all requirements" is the second system effect. It's okay to choose this if you're designing a small system.
"Meets all requirements, and stable" seems to be how most systems are designed. These types of systems are the moneymakers, so if you're doing a startup, it's probably best to choose this option. As Trevor says, it's more like a model train set instead of an elegant mathematical proof. There can be bumps on the tracks, but they won't stop the train from going 'round. Windows is a good example of this. You might think, "Windows? Stable? What an idiot." But look at its size. It's humongous. It's impressive how much it has accomplished, even if it's far from perfect.
"Beautiful, and stable" seems to be the best choice to strive for, once you have the experience. It'll turn out that some of the features you desperately need aren't so critical after all. The Apple II is a great example of this. When it was released, everyone else thought that you needed to sell a monitor with a computer, and that a TV was inadequate. By cutting the need to design a whole monitor, Woz was able to produce something much cheaper and fabulously stable. Also, choosing "beautiful and stable" results in a system that "is beautiful, is stable, and meets all requirements", since you've just trimmed half of your requirements.
From my very limited two years of experience, this seems to be a decent design process for complicated systems:
1. Accept that you don't know as much as you need to know. Look for battle-tested components that other people have already designed, and use those. Copying is not cheating in the world of design. (Give credit where credit is due, though. If you created all your components, fabulous. That's incredibly impressive. But don't claim to have designed everything even if you've made them all work together.)
2. Cut as much from the requirements as possible, and even a little bit beyond what's comfortable. Think to yourself, "How can I avoid designing a new monitor?"
3. Focus on simplicity. Keep the adage "form follows function" in mind. If the implementation is small and simple, it will be far more stable than a big, extensible system that does a lot more. And people will probably like the smaller system better anyway.
4. These are only guidelines. Just focus on doing what makes the most sense, regardless of any rules. (For C++ programmers, this includes rules against using Boost or STL because "it's not fast enough". Take the time to learn your containers, then choose to not use them once you understand them.)
5. Deliver a system when it's 80% complete. (That's 80% complete, not 20% buggy.) Software is powerful because it can be improved later. Draw a sketch, don't paint a painting. The result can always be transformed into a painting later.
What do you think? It's always great to find new things that help designers, so please share your wisdom on all of this.