Software Development: Mistakes I Made
billwadge.wordpress.com
billwadge.wordpress.com
I'm not sure this actually applies to most people. The document vs implement struggle is something I see in modern software shops a lot. It certainly depends on the personality: some people just like thinking through everything and producing a coherent writeup before writing a single line of code, but for some writing code itself is an act that clarifies the still-murky concepts and helps to produce a good writeup.
But in the end, most people aren't producing novel enough technologies that the benefit of a complete writeup is there. You don't write something brilliant and have other researchers work on it; you merely write something for the benefit of a future colleague understand your code quickly. It's a very different class of software development.
This is my philosophy after ~2 years of working on distributed systems. I got sick of vague design discussions where everyone has a 5 minute memory.
I am much happier to sit at my desk and figure it out by writing code, then produce a design doc once I have a solid prototype working. It feels more honest and real.
At the same time, I wonder if I'm missing out on a different way of doing things.
If I'm working within a system I like to code the minimum golden path, add a tests that passes, then add more checks or a new test and go back to the code, etc. Usually the constraints of the system point the way to an obvious skeletal implementation and you can learn as you go along.
On a greenfield project it is especially valuable to have discussions beforehand if people have done something similar before but talking about code at a high level tends to become nebulous rather quickly.
Probably just a mismatch of our testing tools vs the types of things I need to develop usually
If the problem is in a space that we know what the solution 'should' look like, getting as much of that down in the most abstract sense is something I like to do. However that design is only a -proposed- solution, and of course subject to all sorts of changes.
But where things are murky in a design, yeah, sometimes the easiest thing to do is code it out.
In either case, (up to date) diagrams of data flows are one's friend in any distributed system.
For things that are not terribly new, are small enough to knock out with 1-2 devs, and aren't massively complex, just writing some damn code works really well. But the larger it gets the more useful it is to do some thinking beforehand, especially about the overall architecture, technology choices, potentially tricky error cases, etc. Also it can be particularly useful to flesh out public/widely-used APIs and data storage details, as those tend to be difficult to change once put in place.
That said, I don't think a fully fleshed-out "specification" is useful in most cases. A wiki page, or maybe a few, is usually enough. You just want to spot major problems far enough ahead that you can avoid them instead of running into them.
But hey, YMMV, this is just what I've found. TBH I wish it was different, because I actually _like_ hacking out code more. It's just that thinking ahead _works_ better IME.
Most people, of course, overengineer. I remember being on a project where a guy wrote 5000 lines of code to put two columns in a specific browser.
Can relate, every single time I did the opposite, I ended up rewriting anyway because I wasn't following the design doc.
Documenting doesn't have to mean design-first development, it's a way to make sure you're working the right problems and building the desired solutions before you waste your time writing code.
I can't say if anyone else who looks at the code ever also reads what I've written - it seems that generally they don't (even though I announce it and it's in the commit activity stream and PRs).
Agile software development is used by many as a disguise of not knowing what to build. Countless hours have been wasted on "iteration". I'm not saying agile is bad. Just that many are adopting agile without knowing why they should
The icing on the cake was when we were handed evaluation forms he said to the class "It doesn't matter what you fill out because I have tenure."
As for the course , you''re saying I didn't teach for the future. Guilty.
This statement implies that something was taught, what else can help you get an A with your attention
The point is, these two people are incredible Computer Scientists.
That said, when I've seen people over-engineer things (and maybe this isn't your problem, it's just what I've seen most) it's usually because instead of solving the problem in front of them, they created a flexible framework to solve a class of problems including the one in front of them. And it usually fails because they don't actually know what other problems they'll encounter - they guess and get it wrong. So their framework is flexible on Dimension X, but they actually need it to be flexible on Dimension Y.
The best advice I read on this was to never build a framework until you have at least 3 examples of the problem. Once you've got 3 examples, you have a good feel for which ways it needs to be flexible. I would add that you should also make sure you'll eventually have more than 3, because if 3 is the total number of concrete examples you still don't need a framework.
Also, it's entirely possible some smart guy/gal can give you those 3 examples before you even start building. A good product person can do that. If you have that, and they're pretty sure you'll actually get there, then go ahead and build it.
Difficulty of changing it in the future also plays into this, but I've typed too much already. :)
I'm a system oriented thinker. I don't know how I do it, but I'm able to design forward-thinking systems in my head... really well. Like large complex systems. I've since validated this with retrospectives (checking my past decisions, how they worked out, and how they compare to others). The systems turn out to be fairly elegant in their simplicity and ability to scale.
So what's going on here? I think maybe the answer is pretty simple. Some people are good at building large abstract systems and some people aren't. In fact one thing I'm bad at is narrowly focusing on problems. My solutions are always good enough but not great there.
Oversimplified example: if you were to ask me to build a highly performant sorting algorithm I'd probably just give you an okay solution. But if you were to ask me to design a distributed system that runs these sorting algorithms (where the algorithms itself is a black box; implemented already) I'd do really well!