Solving the Wrong Problem (2011)
azarask.in
azarask.in
And that is the wrong problem that too many software engineering teams solve. Let's not solve a customer problem, let's solve the problem of solving customer problems and hopefully in year or so we'll get to that.
Iteration speed is an issue for hardware. It becomes an issue for software teams when too much time is spent tooling, generalizing and reinventing wheels.
I point that out because this sort of example pushes the wrong buttons for many software engineers, including myself. We don't need more time spent on meta-problem solving. Much less, in fact.
The faster you iterate without thinking long-term - the faster it accrues.
20/20 hindsight and all that.
It can be accrued naively, which is an antipattern, but when that isn’t the case technical debt is a vehicle for leverage. You get more value from less work.
With all development efforts having a finite lifetime, and many having a particularly short one, wise exercise of technical debt becomes an essential skill for both project managers and developers.
Vilifying technical debt, and even prematurely mitigating it, are just as naive as inadvertently introducing it.
Holistically, if your system is meant to endure the test of time (think Google-scale), more technical debt is worse than less technical debt.
While some exceedingly few products have that destiny, it’s precisely the naive bias towards thinking that your project does that leads to wasteful over-engineering.
Products and projects are not the same thing. Most of the Google-scale products that you see leveraged tons of technical debt on projects along the way, some critically valuable and some certainly wasteful.
If you’re adding a feature to one of these products that are already at that scale, then your attitude toward technical debt will be different.
But effectively nobody reading this thread is doing that, and the few that are know who they are.
Most readers here are working on projects with near-term deadlines and development lifetimes measured in a few months or years. As much as judicious exercise of technical debt enables those projects to deliver on their requirements more quickly and for less cost, more is better.
Many of us come into the industry thinking otherwise because we’re drawn to the intellectual purity of clean systems, but that purity just isn’t what actually matters most of the time. It can take a while to accept and internalize that.
That's the false dichotomy which earned me my stripes.
I used to think I am the former guy, then I realised I am the latter guy. The only thing that changed is that some time passed.
The hind-sight of my mistaken identity is that the systems I have been working on for 15 years became less, not more maintainable as a result of tolerating technical debt.
They also became Google-scale (which is what we were hoping for, but didn't believe at first).
I am the furthest thing from a purist. I am a filthy rich pragmatist who regrets having low-quality standards.
We'd all like to be a filthy rich pragmatist but I wonder just how much the 'low-quality standards' enabled you to achieve that. Seriously, I would love to be a millionaire with regrets about low quality code instead of being broke with junior developers pointing out my code smells and technical debt.
These kind of assertions are always after the fact. And in hind-sight I think we could've done a tad better.
I might be less-rich, but that's not even close to being broke. And the people I hire to clean up my technical debt would hate working here less.
I've always felt we've misunderstood technical debt. It seems to be regarded as a problem with a lot of negativity surrounding it. Developers seem to fear being accused of introducing technical debt - like it's the worst kind of developer crime. But in reality, there is value in embracing it in early stage projects. As thinkingkong mentions below; 'Its only debt if it sticks around long enough to need to be dealt with.'
> But effectively nobody reading this thread is doing that, and the few that are know who they are.
Aren't a large fraction of HN readers working at Google or similar companies?
Adding a load of automated tests is the first thing we need to do start being able to iterate more quickly. (I view this as both paying down technical debt and solving the right problem). Thankfully that was discussed just the other day in our tech debt meeting (while the tech lead was saying how we need to add a Neo4j database, a Leucene search index and remote procedure calls - he had read an article saying how that is cool again).
That's exactly what causes too much time spent on tooling and reinventing wheels. Not that tooling is bad, but often you spend two weeks in order to save one hour. And that is the best case scenario, where the extra complexity usually will slow down future development - rather then make it faster.
In the article MacCready strategy was fast iteration, rather then focusing of safety and engineering. Basically the "go fast and break things" motto.
Imagine you have to build a word processor in assembly language. At that point it may be worth it to build a compiler first.
Being able to analyse existing solutions and previous attempts (successful or not) is a luxury that the first people to attempt a solution don't enjoy.
It's much, much easier taking an existing solution and improving it than it is to develop the existing solution.
He talked about the tendency really good engineers have to spend a lot of effort optimising the design of things that shouldn't be there at all.
Finally he banged on hard about always questioning your design constraints. He pointed out that if you get given some design constraints by another team, you should always consider their validity since it;s very unlikely they are optimal. The converse assumption would be that they are always perfect, so looking at it that way there's no reason you should assume they are right from the start.
For anyone who worries about the sunk cost fallacy, bear in mind it was only about a year ago they decided to throw away all the investment they'd put into building a composite body, including a huge main body tool.
In that one interview with Tim, he explored very deep concepts about cross team collaboration, engineering process and project management that extend well beyond the rocket science I was expecting to hear about
Teams chasing after arbitrarily defined KPI, with no feedback loop in place, or the feedback is 1 year later.
Implementing features based on feedback from 1 customer, then find out the customer wasn't articulating the problem at all.
Projects eager to adopt the latest technology, using every bit of service AWS provide, jumping onto the jargon bandwagon (e.g. blockchain) to solve non-existent problem.
To name a few
What about, ahem, Google Self-Driving Cars :)?
Unpopular opinion around here but here goes:
I feel the dimensions on which actual people (i.e. customers) will evaluate Google's SDCs, when they become generally available, is very different from the dimensions Google is optimizing for, based on what they think people will need in the future once the technology is ready§.
How a person determines the utility of a product or service can be markedly different from how the same person evaluates the utility of that product or service when s/he is confronted with what all potential customers must confront before making a purchase decision: a price tag.
As subtle as the distinction between a person and a customer is, it is an important dose of reality that I think is missing from a lot of arguments that happen here on HN where the title of market leader is undisputedly ceded to Google Waymo when compared to a competitor like Tesla with far far lower-tech (but cheaper) autonomous capabilities.
§ IMHO, I don’t think their current approach to SDC technology using expensive LIDARs fits the problem at hand. Use of high fidelity tech like LIDAR also causes another tendency, it lures engineers into thinking that it will somehow make tractable, a hard problem like the trolley problem [0].
An additional issue is that the designations: L0-L5 are arbitrary constraints themselves, they are not based on problems customers are confronted with.
> Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.
John Tukey
"It is better to be vaguely right, than precisely wrong."
- Carveth Read, "Logic: Deductive & Inductive" 1898 (1920 4th Ed.)
[1] http://www.gutenberg.org/files/18440/18440-h/18440-h.htm#Pag...
http://content.time.com/time/magazine/article/0,9171,90512,0...
MacCready's engineering insight, on the other hand, arose from an understanding of the practical implications of aerodynamics so thorough that it was almost instinctive (the same could be said for his invention of the glider speed ring; it is not that he knew something others did not, but he saw how that knowledge could be put to use. Also, in both cases, there's some significant engineering to be done between the idea and the implementation.)
This aerodynamic insight happened to open the door to fast iteration, but, perhaps more importantly, it showed what sort of airplane to build (in fact, the latter was the key to the former, as well as the key to actually achieving the goal.) It is all very well to say that we're going to solve a problem through fast iteration, but that does not tell you what to do next. On the other hand, I imagine that as soon as Dr. MacCready had his aerodynamic insight, his mind was filled with ideas about how to go about it.
https://en.wikipedia.org/wiki/Paul_MacCready#/media/File:Pau...
The example of Feynman cracking safes is perfect - he just used a bunch of special tricks rather than any general method - similarly with quick numeric calculations in his head.[1]
That said, while most hard problems need to framed in a different way not all hard problems yield to the "clever detour into something simple" approach. A lot of math problems are solved by a detour into a bunch, difficult things and then a return to your original "seemingly simple" problem (see Fermat's Last Theorem etc).
[1] See: Surely You're Joking Mr. Feynman https://en.wikipedia.org/wiki/Surely_You're_Joking%2C_Mr._Fe...!
A bit more in 2014: https://news.ycombinator.com/item?id=8524988
Quite a bit in 2012: https://news.ycombinator.com/item?id=3561397
Discussed at the time: https://news.ycombinator.com/item?id=2591367
there aren't much right problems in software development to go around, while there are so many people and money that can and have to be put to work, and as result we live in a kind of a golden age where instead of subsistence and survival of a typical stressed&oppressed office drone most of the people in the industry are in a position to work on wrong problems, and to try and to develop more and more of new tech, and the high failure rate of the projects isn't an issue. Which other engineering discipline can allow itself to have such a luxury?
There are two ways of doing a Ph.D. thesis: (1) Finding a problem and solving it; (2) finding a method and then finding a problem it solves. The second is much more likely to be successful.