Practical guide to solving hard problems
praeclarum.org
praeclarum.org
1. Tightening feedback loops as much as possible. This means both recursively breaking down the problem into subproblems (as discussed by the author), and having very fast ways of doing formal testing, sensitivity testing, etc. Sometimes I'm working on a problem where I have to solve a system of equations that takes a few hours. This means that progress is really slow and I just won't be able to turn all the knobs that I want to or try alternative approaches. If I can subsample the data or the model domain down to 20 minutes, then that gets better but I still need to do something else during the downtime. If I can get it down to 2 minutes, then I can really make progress and if it's a few seconds then I can fly.
To do this I might have to invest a few hours or a day into making a testing dataset and environment that allows for very rapid iterations (but may not be useful once the particular issue is addressed), but I am often averse to doing because it feels like a waste of time, even though my velocity averaged over the month is usually higher if I do it.
2. Work on the hard problems first. This is in contrast to what the author suggests but I would rather take the bull by the horns knowing that once I address the hard parts it's going to be a lot easier. If the hard parts loom in the distance I worry that I will run out of time, because even on the easy parts it's possible to spend all of the time making small tweaks. Do the hard parts, or at least a draft implementation, first so you can allocate your time over the project more efficiently and ensure that you're not going to blow up when you're 80% done but just hit a massive roadblock.
Related things:
A book that is very digestible and praised is "How to Solve It" by George Polya (Original: "Schule des Denkens. Vom Lösen mathematischer Probleme"). It has focus on math teaching but many of the principles are generally applicable to problem solving and teaching.
A talk I really like is "Hammock Driven Development" by Rich Hickey. Very enjoyable and goes into some of the longer term things we can do around problem solving. Utilizing time and letting the brain work in the background.
I just a few weeks ago had to refactor and generalize some hairy part of a web app. I could do it well because I was thinking about the problems over a longer period. In the end I frantically hacked away the solution and now have a nice little extendable library that is quite a bit more useful than what I had before. I even _found_ and _fixed_ some bugs while sleeping - just popped into my mind after waking up at some point.
You might find that the problem you were planning on spending weeks thinking hard about was solved thousands of years ago by a reclusive greek guy. Or that a couple Nobel Prices in Mathematics have spent 10 years thinking about how to solve it, but are not there quite yet.
But just a digression, if solving the problem is not that important like with hobbies, it is usually more fun and more rewarding to solve the problem without googling; like playing a video game without looking at a guide beforehand.
Q.U.E.D.
For instance I often find solving problems with CSS 'hard' and that is some combination of the complexity of the environment (the web browser), my own ignorance, and the tendency for the code I'm working on to have been developed based on the "look it up on Google/Stackoverflow" such that I don't have a solid foundation to work on.
Another kind of 'hard' problem that I relish are problems that are hard because of circular dependencies and I find it really rewarding to go over the problem again and again until I've found the irreducible core of circularity, build a minimum-size system that resolves it, and then build the rest of the system as a set of clean layers on top of it, often using techniques such as code generation.
I have always thought the "subconscious mind" was a bit of a myth. But the more I have learned about it, there is a lot to be said for having some simple unstructured time to let your mind process. No music. No podcasts. Just a walk.
Alternatively if I start tackling the problem immediately and really put my nose to the grindstone there’s often a lot of moments where I realize I didn’t understand the problem fully, and waste a ton of time backtracking. So it ends up taking the exact same amount of time, only difference is how many times I banged my head against the wall in frustration.
If you really think about it, aren't all insights subconscious insights? Even if you're consciously stringing it all together, there's no thought process for adding two and two. You just look at '2+2' and out pops '4', ready for your conscious mind to use.
I had an insight recently about this, because I also have a tendency to trust my conscious more than my unconscious.
What is the conscious made of?
sorry if that gets too zen/koan-y.
This will lead me to trying to figure out where my knowledge gap is so i can fill the gap and then try again to solve the problem
[0] https://jeremymikkola.com/posts/2022_01_01_a_few_notes_on_pr...
> Think hard about the problem for a few weeks before typing any code.
In my case, it's never more than overnight, but maybe it's because I am working on a smaller scope than the author.
You’d be surprised how experience can shorten the planning time.
Also, I’m not quite “neurotypical,” and avoid writing things down. I tend to keep my strategic model in my head. I’m constantly wargaming my strategy. You might be surprised at the scope of the project I’m doing right now. I’ve been working on it for around two years, and we’re getting within sight of the finish line. About the only time I write stuff down, is when I want to pass it by other team members.
I’ve also learned that, sometimes, there’s no substitute for writing some code, stepping through it, and seeing how that jives with your strategy.
To do things my way, you have to be willing to toss out a lot of code; sometimes, weeks’ worth of code.
If you wasted weeks coding the wrong thing then you didn't solve it overnight. Making mistakes is a part of the process, you have to count that time as well.
That was what the OP said.
For me "typing code" is part of the thought process. These days, with GUI IDEs, and symbolic debuggers, there's almost no excuse to "measure twice; cut once," like we did in the "big iron" days.
I was trained as an artist, and the process of sketching is a lot like the way I work in architecting my code.
Also, I wouldn't call it "wasted" code. What was it that Edison is reputed to have said about 10,000 failed experiments?
"I now know 10,000 things that don't work."
I actually enjoy coding. I also really like my code to be clean, efficient, and useful. If code that I write is not all of these, I don't want it in my work.
I saw a chap that had a .sig that said something like:
"I hate code, and want as little of it as possible in my programs."
Few things make me feel as good as ripping out large swaths of code. It's like lancing a boil.
With coding, you are confined to formal syntax and semantics, but if the code (even partially) works, you can be more confident in your design.
With paper, you can plan as high-level as you want, with the danger of being too highlevel and overlooking things.
I cut my teeth, in the days when we were supposed to design the entire program; from start to finish, on a pad of paper, hand it to a data entry clerk, who would then create a deck of cards, based on the work.
It would then be scheduled for an expensive slice of time, and, if it screwed up, you got spanked.
It sucked. It really sucked.
Full disclosure: by the time I entered the field, punchcards had been replaced by VT-100 terminals and line printers, but the process was still the same, minus the data entry clerk.
These days, it's totally freewheeling. I try stuff out, screw up, kick myself, then try again.
I write about how I do stuff, here: https://littlegreenviper.com/miscellany/thats-not-what-ships...
What happens to the project if you get hit by a bus?
It’s over (but maybe not -see below).
Sucks, but that’s life.
This is a nonprofit, and they can’t afford to hire a large team of programmers. Heck, they can’t even afford me. I work for free.
The code, itself, is documented almost beyond belief. The only place I’ve ever seen documentation that even comes close to what I do, is in exposed Apple and Adobe code (where I got a lot of my inspiration). Don’t believe me? Look for yourself. My GH ID is the same as my HN ID[0]. All my code is like that. Look at any one of my projects. They are all heavily documented, and loaded with meaningful, relevant tests[1].
I’m a big believer in what I call “forensic design”[2], as well as really heavy-duty documentation[3].
So that means that, if there’s a developer with skill like mine, they’ll be able to take over the project in a relatively straightforward manner, but less-skilled people will probably be fairly stymied. They are likely to complain about my code, and rewrite from scratch, with typical modern crap code (and with typical modern crap results).
I’m not gonna do a crappy job, and make dross, just so some inexperienced jargonauts can mess up the project in the future. If that means it dies with me, then so be it. I’ve gone way beyond the pale, in delivering a project that can be maintained. If the people I deliver it to, mess it up by cheaping out on the hired help; that’s on them. They are the ones harnessing a racehorse to a plow.
I have a fair bit of prior art. I’ve written software architecture that has lasted decades, and has been taken over (and is still very much in use), by teams of skilled engineers. I don’t know if it’s still in use, as it’s been five years since I left my last company, but, at the time that I left, if you got their DSLR SDK, it had a C code API in it that I wrote in 1995.
[0] https://github.com/ChrisMarshallNY
[1] https://littlegreenviper.com/miscellany/testing-harness-vs-u...
[2] https://littlegreenviper.com/miscellany/forensic-design-docu...
[3] https://littlegreenviper.com/miscellany/leaving-a-legacy/
Reaching high performance means knowing the solution before coding it up.
Witnessing high level competitive programmers scribble stuff on paper for 5-10 minutes, and then just a pure stream of thought into a 100 line C++ program that was compiled once, run through examples, submitted and accepted, is out of this world.
Another example is when solution is unclear (mostly for number theoretic stuff that needs to be efficient), scribble on paper, code up pattern generating code, generate patterns, look at them, stop, scribble on paper, code the O(1) solution, submitted and accepted (an easy example is to efficiently calculate the sum of first N Fibonacci numbers).
I'm not competitive, anyway. It's really a life choice, and a moral stance.
Also, that's not how my brain works.
I'm not really interested in proving myself to anyone, or beating anyone else.
I just enjoy writing good stuff that people use.
I'm always humbled when I try to solve something, need 50-100 lines of code, and then there's a solution that really gets what the inputs/outputs/intermediate data structures are and it turns out to be just 10 lines of code (written during live competition, I'm amazed by the truly simple thinking of these authors). The best problems that demonstrate that are most often just algorithms with arrays (no specific algorithm, only required knowledge is working with arrays). True display of thinking things through before coding.
I guess example that come to mind (a mindblowingly simple algorith, try to find it yourself): https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_majority_v...
It gives me similar feeling (but less pronounced) to looking at tinycc, or looking at some tree enumeration algorithm in TAOCP, or the modification of it done by Knuth to support enumerating arithmetic expressions without redundant parentheses (also just an array algorithm, figure out how to modify an array that represents existing expression into an array that represents next expression, literally just a bunch of loops).
I had to handle time series data at work. There already exists 1000s of lines of code that deal with time series, sorted timestamp arrays, {up,down}sampling, timezone handling etc. All of that can be replaced by a simple 5 line function that works faster (supports tz, sampling, appends etc.). Only possible to write if you really focus on the problem you're solving and realize that the sub-problem you envisioned (the "efficient" time series data structure) does not really help you.
to me its critical to nail the hardest part first since if that portion cannot be solved or needs a radical rethink it will often require entirely new surrounding logic
On the other hand, when I don't find such a function at least I know I'm solving a new problem.
step 2: Challenge assumptions about implicit requirements, the scope of applicability of explicit requirements, and boundary conditions.
Then, in some magical moment, a loophole opens, and a solution manifests. I love that feeling.
I wish there were more such hard but then not so hard problems.
Now it may be that it's my job to specify it as well as implement it, which is fine, but those are two different problems.