Why Programming Is Difficult (2014)
joearms.github.io
joearms.github.io
> This problem is, by the way, the thing that takes most of my time, 60-70% of my time at a guess. I once spent over a week trying to get a broken LDAP server to work - my boss had forbidden me to implement my own LDAP server - but after a week of struggling with a broken LDAP sever written in C and badly documented I had a lapse of memory and forget that my boss said and accidentally implemented a server from scratch in Erlang during my lunch break.
> To be honest it wasn't a full LDAP sever, but I didn't want a full LDAP server. I only wanted a couple of commands to work, and that was pretty easy to fix
Knowing when to flip the table and reimplement is probably more challenging than anything else outlined in the article.
Joe probably had a good sense for that. Most programmers, however, do not seem to (requires tremendous breath of experience!).
It takes a certain amount of arrogance to assume that you understand (or can predict) the entire problem space along its warts. And if you're wrong, it's often another poor programmer down the line that pays for it—not you.
This is an eternal problem of software design, when do you borrow, and when do you implement your own. I don't know the answer but I think there are some factors that can lead you in the right direction.
Did you do your homework? Implementing your own tools should be the last resort not the first resort. It should be driven by frustration at the inadequacy of current tools, and not based on a whim to do something cool.
Of the examples tools you've found, how much customization is required to get them to do what you want? If a 3rd party library requires a lot of hacking to do what you want, it's a hint that those tools don't actually solve your problem and you're going to end up hand rolling what you really want anyways.
How much overhead do the other tools bring in? Sometimes you need to do one simple thing, but all the tools available have a lot of overhead. That's a hint that maybe you should do your own implementation
Have you talked to other people about this problem? A conversation with a capable colleague is usually the fastest way to get useful perspective on what exactly you're trying to do.
How well do you understand the problem space? No matter how much it may suck, if you don't know what you're doing, take the time to figure out how to use the tools instead of creating your own.
All of these factors must be balanced against each other. One of the most important things though is that if you do decide to hand roll a specific tool, don't keep it to yourself. Such pieces of code can be a little hairy sometimes and you need the feedback from others to ensure the code is clean, and to make sure they know what's going on.
IME having to learn a fancy framework to edit a couple of things on an open source app that was mostly what we needed hurt a lot more than it would have to make something very KISS with something like Flask as a proof of concept.
Whether choosing to implement your own thing from scratch should be a last resort, or that it necessarily requires tremendous breath of experience, really depends on the circumstance, as you described. In many circumstances you don't have to be a genius to know NIH is probably the sanest option.
Back around 2000, I installed tftpd so that I could boot a VxWorks board. It came locked down for security to the point that it wouldn't even work, and the documentation was no help in making the tftpd operable. After much frustration, I realized that writing tftpd with plain old C and sockets would be easier. I did that.
A decade later, I again needed TFTP. Surely the passage of a decade would have sorted out the issues. Nope! Having long ago lost my reimplementation due to job change, I very nearly wrote TFTP yet again. This time I was saved by the discovery of hpa-tftpd, which actually works.
I wonder why hpa-tftpd is not the default or only TFTP server packaged with Linux. The regular package is, as far as I could tell, non-functional.
Actually, I have no problem reading the 391 page manual first because I've learned, the hard way, that that's the best (only) way to make sure you're not overlooking something - it's my boss who has a problem with it. Thy fingers shalt be on thy keyboard at all times and thou shalt be closing Jira tickets as fast as possible.
When I try and learn something and then get to the point where I know what questions to ask, that is when I know that I have learned something.
If you’re staring at a problem and you don’t know what it is or how to describe it then you can not progress, but that is how you progress. You try and fail and ask yourself why you failed. Building on failures is a slow way to progress, but gradually those failures becomes knowledge and it gets easier to progress.
When I learn a new language, tool, library or feature my knowledge gets me going quicker. It doesn’t necessarily bring me the whole way home each time, sometimes you really have to learn something new, but looking at that thing that I know that I don’t know is what gives me confidence in feeling that I will learn it.