Solve many different problems, make experiments and throw away code. Make connections between sub disciplines.
You have to have the attitude that if somebody understood this, then I can too, if nobody did, there's no good reason why I can't be the first one.
Fail a lot and learn. Think about every failure you had, but as a problem to solve not a comment on your value as a person/engineer. What could you have done differently? What were the signs that got you in that situation?
For tricky problems, what works for me is the obsess-and-let-go strategy. Work intensely on some problem for some time, if you make no progress, just let go of it, forget about it, do something else and perhaps your brain will connect the dots. Talk about the problem with other people. Explaining it and different points of view usually change your perspective enough that you are no longer stuck.
Also note that reading about something is not the same than doing it. You need to read and attempt to replicate, even if a toy version of the thing to better understand. Some things just take time an effort to seep in.
2. Keep asking 'why'. Make sure you understand what the problem (or rather the requested feature) is, instead of being able to parrot what someone else says the problem is. (Even if the conclusion ends up being the same, you need the gained comprehension to determine the right solution.)
3. Separation of concerns. Keep practicing on separating orthogonal concerns that are all seemingly relevant to a problem. Evaluate each concern separately. Drop any of the concerns that are irrelevant in solving the solution. This can save you huge amounts of time, enough to make a deadline you would otherwise never make. (This will also help you to reduce the amount of added complexity.)
Cobble together an ugly and hacky solution that you can confirm solves the problem, then iterate and iterate until you clean it up well enough to ship it.
Make it.
You can't make it work if you haven't made anything.
You can't reliably deduced anything about what your current problem is, if the things it depends on are flaky.
Design by Contract. Understand it. Used it. Turn your asserts on always.
Turn on all warnings that don't give false positives that your compiler / tools provide.
Fix them don't mute them.
Run around your code base and find every damn place where some damn fool has discarded an error code and check and report all failures.
You'd be flabberghasted by how many "it's impossible" bugs I have found and fixed by that one simple action alone!
try:
something() # that throws nonfatal errors frequently
except:
pass
Ugh.
Don't be like that.
If you don't know the precise error, heres the solution (in python):
except Exception as exc:
log("{}: {}".format(type(exc).__name__, exc))
Interesting, as a junior dev I always start thinking about the fun architecture stuff first and end up being slow to complete a project. I figure that deploying one place vs another is so different you want to deploy early and often for testing. Maybe I should just start getting the basic functionality and tests down before packing it up into docker containers and running it on kubernetes and all that?
TDD to get the chunks of functionality you add rock solid..
A Walking Skeleton to guide your vision of where you're going... https://wiki.c2.com/?WalkingSkeleton
Each part you add must be rock solid before you move on to the next that depends on it.
That doesn't mean you have to do everything... merely make it solid and reliable.
Say you plan to depend on a subsystem that is going to do something wonderful and flashy.... but you don't need it right now, you just need to build the place where it's going slot in....
Then don't put your sketch of the wonderful shiny thing into your dependency tree, put a mock that might do nothing, but does nothing utterly reliably.
If you have something flaky firing bullshit events and rubbish into your whole system while you're trying to get the rest going... you're just going to hurt yourself.
You can add a feature toggle or chicken bit to switch the new shiny/flaky stuff in when you want to contemplate it, and switch it out when you're working on the rest of the stuff.
Be explicit with what phase of your "thinking" you're in. When faced with a problem, focus on giving lateral/divergent ideas and come up with a multitude of different solutions. Don't spend time thinking why/why not a solution is good, just note the solution itself and move on to finding another one.
Once this is done, start the next phase which is the convergent thinking. Sit down with all the different solutions and start compare the tradeofs between them. Eventually you'll reach either some middle ground of some solutions, or just one solution.
Doing this when collaborating with other engineers is also super helpful, as people won't feel like their ideas are being shut down as soon as they utter them, so more people can part of the whole process (if needed)
https://www.hallaminternet.com/how-to-think-creatively-using...
No one will ever successfully transfer the problem in their head to your head first try. You will need to ask a lot of questions to try and bring your understanding near their understanding.
Equally you need to actually understand why the problem you're solving is a problem. If someone wants the header on their website to be bigger, that's a very easy thing to do but there's always a reason why they want it bigger and knowing that will help you do it properly.
Also, always ask why whatever solution you came up with works. There are very few situations where the actual answer is "Invoke these magic runes in the right order and pray". Spending the time to find out what the magic runes are and what they do will help you make sure that a) The problem won't come back b) Your solution is actually a good one c) You can solve similar problems in the future.
Realizing that technology and architecture ultimately doesn't matter that much for 99% of the use cases and sticking to proven technology to deliver value (postgres, rabbitmq, ansible, redis and django)
When coming across a problem, I type out all hypotheses on a notepad.
Microphone doesn't work on app? Is it this device specifically? Is it the file system? Faulty recording code? Null pointer somewhere?
Then I find the simplest, hackiest way to (dis)prove the hypothesis. Be sure to narrow down the test as far as possible.
What a lot of people do is just bash random things, often the simplest hypothesis, repeat that same thing for half an hour, do something else, then repeat what they tried earlier. Or in somewhat tougher problems like pathfinding, where each hypothesis takes an hour to test, a lot of people get intimidated and procrastinate.
Interestingly, "eventually" usually means in the shower the next morning. From that I assume that I'm more receptive to flashes of inspiration while I'm relatively relaxed in the shower, and also that the real work of finding a solution happens while I'm asleep.
Ask and agree on the "why" behind requirements, then the what or how to realize it becomes more easily negotiable.
As a result of being burnt, I'm happy to give up on logging and trawling through code in favour of just taking a PCAP and finding out what's actually going on over the wire. Or stracing my app written in a high-level language which runs in a VM. Sometimes you just want to see the syscalls.
I'm also happy to go digging in the browser's source code. I think my favourite bug to diagnose manifested as a visual issue with menus in a frontend framework. The menus were styled with some CSS, nested inside a media query:
@media (hover: hover) {
Getting to the bottom of this involved creating a test page to reproduce the problem, reproducing it in multiple browsers, digging into Firefox for Android's source code (yay FOSS) to find out how it implemented the media query, writing an Android app to reproduce the underlying data problem and eventually working out that it was a problem in the phone's operating system.
Certainly being able to trace code is one skill, and knowing how deep to go is another.
It's very clever that you managed to discover a bug in OnePlus's version of Android but then what?
I've done this sort of thing for non-web problems
It enabled me to go back to the customer and explain the problem and make the informed decision to not continue work on the problem. The fact is that the broken OnePlus devices (not all of them, and not all the time) represent a very small proportion of the devices we need to support.
confirming fundamentals is another tried and tested method..
other than logging and methodology, having the right tools is important.. I use pudb for my debugger in python.. and bash -xv for shell..
error reporting is also important.. I use 'trap' and set -E in bash to capture all errors, provide a lot of info and even email it to me..
in python I have exception hook.. I even monkey patched threading to report its uncatched errors to the main threads exception hook (I use python2.7 for work.. this is fixed in latest python3)
during my dev phase I always give more time to coming up with a solution vs with running with an idea.. this allows you to engineer a solution..
and lastly.. in my down time I think about the problems I couldn't solve at the desk.. this could be when I'm about to sleep.. when I'm playing pool.. etc.. I find that solutions come easier at these moments..
It made me realize that focusing on the flows of data between systems resulted in better overall designs. You can always swap out the contents of a box later on.
If you can't find a listener just find a place where you can talk to yourself quietly.
Basically offer up a detailed commentary of the problem and what you have tried so far. The slow, methodical pacing of actually speaking out loud, as opposed to mile-a-minute thinking in my head seems to help my brain spot problems and come up with solutions.
When dealing with some of these it's often necessary to delve deep into the internals of how something is working. I once spent 4-6 months working on a fix which turned out to be a stack corruption on a RTOS but was hard to pin down as the system usually took over 2 weeks to crash.
I now mostly deal with embedded software so it can often be with conflicts of hardware which have never been tested together and with unpredictable results. I've also dealt with finding faults in software I had to maintain across several platforms/databases/etc written in a few languages.
For the most part it's been fun or rewarding with some difficult times. Delving into the depths of a system does take some time but gives a wealth of knowledge that can usually be applied to another problem
I've got a tongue-in-cheek presentation entitled 'How to seem a Genius at Debugging with this 1 Weird Trick' based on decision theory. One day I must write it up properly as a blog post.
- In bayesian search theory a good (sometimes optimal) approach is to maximise information gathered per unit effort
- In debugging, a step can vary by several orders of magnitude in amount of effort required to check a theory (eg, from 'type ls' to 'spend several hours implementing a theorised fix/tracking framework/etc)
- Consequently, the probability of a hypothesis can be quite low and still be the optimal thing to check first, if the effort to test it is also low.
So, the 'one weird trick' is to look at all the evidence you can look at very quickly, even if the connection to the problem is tenuous. 95% of the time the tenuous ones don't pay off, but when it does, you're colleagues are like 'WTF? how did you think of that?' and as advertised, you look like a genius.
In terms of smaller strategies:
I think stepping back and asking "What is the real problem that needs to get solved here?" is very often the best thing to keep in mind when I'm stuck. Along with "How important is this, really?" These kinds of questions help you to evaluate whether a particular technical roadblock is actually worth solving, or whether perhaps it is not worth the time investment because some other solution (or just walking away) would be a better choice.
Also, I don't know if you need to hear the clichés, but context switching is a huge and important strategy for problem solving — whether rubber duck debugging, talking to colleagues, switching to another medium for thinking (like a white board), or just going for a walk around the block — these are often good ways to get unblocked.
- Never stop learning! Luckily, nowadays we have plenty of resources(HN, technical books, podcasts, etc...) we can use to improve Domain Knowledge on a subject, this is extremely important because it will create the hypothesis set we can query to solve problems later.
- Being resources constrained. Working with too many degrees of freedom is not only a common UX antipattern but also a way to kill creativity. If it's possible in your company, I strongly recommend doing technical customer support twice per month. Imho, this is a great setup to improve your problem-solving skills.
I do the job with person, to workout what needs changing rather than them telling me. This saves hours of back and forth and people rarely do it.
The wealth of knowledge, experience and advice here is pretty unique. I always learn some new way to think about a problem or new actionable things to try every time I come here.
That and just reading. Honestly. Read anything.
https://www.amazon.com/Are-Your-Lights-Gerald-Weinberg-ebook...
Also, working on legacy systems, and seeing why and how the theoretical drawbacks of some approaches manifest in practice, and other theoretically discussed risks really don't.
all the normal pain points are there: proprietary standards, 16 standards, major tooling fights, bandaid fixes that “we’ll refactor later”, tech debt, etc.
But the architecture is really really visible. And when you’re done you have a bike!
Doing leetcode like problems on a semi-regular basis helps me a lot.
This skill wont solve all of your problems, but if you have a lot of theories you want to test then your ability to implement them quickly is invaluable, and testing a lot of your theories is how you learn most of the other important skills related to software engineering.
There are no coincidences, unless proven otherwise.
If something smells wrong, it probably is. Trust your gut.
Make sure you're building the right thing, before you build the thing right.
Don't be clever. Elegant one-liners that make you feel like a genius when writing it are probably not very maintainable.
The second best piece of code is the one you just deleted. The best one is the one you didn't write in the first place.
Plan to fail, and gracefully degrade.
1. Look at prior art. Many problems have been solved before. It can be more fun to dive in and create your own solution, but looking for previous solutions first often saves time, results in a better solution, or may show you that you don't need to write the code in the first place.
2. Write down the problem you are trying to solve before solving it.
3. Solve the problem multiple times before committing to a final solution (if you have time). Unless the problem is familiar, your first solution will not be the best one.
4. Learn when to ask for help. This depends on the scale of the problem, but in general if you aren't making progress within half a day, consider asking someone else for their help and ideas.
5. Do rubber duck debugging. People who talk through problems aloud are better problem solvers and learners. https://www.teachervision.com/problem-solving/think-aloud-st...
6. Make documentation easily accessible. Some recommend spaced repetition like Anki (https://sivers.org/srs) to recall library functions and patterns. I've found that just having locally cached documentation in an app like Dash (https://kapeli.com/dash), Zeal (https://zealdocs.org/) or Velocity (https://velocity.silverlakesoftware.com/) to be a big help, especially if you don't just work with one language every day.
7. Do programming puzzles. I've found it helps a lot with software maintenance and other problem solving at the micro level. (It doesn't generally help with big systems design because puzzles tend to be smaller in scope than that, but a lot of our work is maintaining existing code.) I find https://exercism.io/ the best for this because you usually complete puzzles in the same environment you'll use for daily work (your text editor/IDE of choice), rather than in a web-based environment with limitations.
8. Consider courses like HtDP (https://htdp.org/2018-01-06/Book/ ) and the Clean Code series (Clean Code, The Clean Coder, Clean Architecture) for basic advice on structuring systems. Find others at https://mustread.tech/books
9. Make things. Keep them small in scope to begin with so that you finish some of them.
10. Surround yourself with brilliant people. Feeling like the dumbest person in the room is intimidating but you learn more.
11. Read and learn outside of the field of software. In the words of Feynman, “everything is interesting if you go into it deeply enough.” I like In Our Time (https://www.bbc.co.uk/programmes/b006qykl) and https://www.thegreatcoursesplus.com/ and https://www.masterclass.com/.
12. Look after yourself. Eat well, sleep well and exercise.
What do you think helps specifically with that?
Talk to other people who build “big systems”, whatever you decide that definition means. Ask them lots of questions. (Not the “Emacs or Vim?” kind. Instead: “When you wrote x, where did you start? How did you arrive at the solution you found? What would you change if you started again today?”)
Build systems, big and small, on your own and with others.
If you're not sure what to build, I like “Exercises for Programmers” by Brian Hogan:
https://pragprog.com/book/bhwb/exercises-for-programmers
The problems are still pretty trivial but a little larger in scope than most from https://exercism.io.
Beyond that, I try to find videos or tutorials of people solving real-world problems. For example:
Go: https://gophercises.com/, https://gowebexamples.com/, https://www.usegolang.com/, https://lets-go.alexedwards.net/, https://testwithgo.com/
C/C++: https://handmadehero.org/
Haskell: https://typeclasses.com/
Mixed: https://howistart.org/
Pair programming and live coding videos on YouTube can be worthwhile too.
“Hacking with Andrew and Brad” is an example for Go: https://www.youtube.com/watch?v=yG-UaBJXZ80
“Pair Programming with Jon Skeet” is another example: https://www.youtube.com/watch?v=zvz2omT9fzk
You get to hear people talking through their thought process and learn how other people solve problems and structure solutions.
The Twitch Science and Technology section is another place to find similar stuff:
https://www.twitch.tv/directory/game/Programming
I like Tsoding for functional programming:
And SerpentAI for Python/AI:
https://www.twitch.tv/serpent_ai
Watching people code can be dull or exciting depending on your mindset. I try to dip into these channels occasionally and soak up ideas from them, then go away and make things. I don't generally learn much without writing code too, and look for technical books and courses with exercises and problems.
We need more of this, though. I wish there was a programming puzzle site full of problem statements of the type, “build an application that does x”, where solutions are ranked by the community and sorted by language. It would be such a helpful resource.