> So it was not so much about any technical debt, but rather about the lack of understanding of what the client wanted (often the ignorance of the client himself of what he actually wanted).
I cannot emphasize enough how much I want you to read, watch, listen to Ward Cunningham in his own words. :)
If you know what technical debt really is, then this sentence kind of sticks out as a sore thumb, as it is literally saying "So it was not so much about any technical debt, but rather about the technical debt."
Technical debt viewed this way is not just “hurdles in my code that I don’t like to jump over.” It is a subset of those hurdles and kludges and nicks and grumbles. What subset? Precisely the subset that corresponds to our lack of up-front understanding of what we were building when we were at first building it.
Put another way, every single problem we can solve well has at least one (maybe many) perspective from which that solution seems “relatively straightforward.” If you can specify that perspective in a way that a computer can understand it, you can think of that as “creating an algebra” for that problem and similar problems, or perhaps a “domain-specific language.” Technical debt is precisely the mismatch between the language you are programming in, and the domain you are programming for. Syntactically, this "language" might be some programming language like Java. But semantically, you get to define your own classes and methods and create your own little world inside that language, where things can be combined and chained together, and the richness and algebraic predictability of that little world informs the spaghettification and robustness of your code to implement the algorithm within that world.
Viewed this way, a lot of programming today spends absolutely no investment into that "little world" and just consists of writing FORTRAN. Just pages and pages and pages of FORTRAN. I mean, you're writing it in some other language, maybe Python, maybe Java: but you do not have any sort of little algebraic world that you phrase your problem in, you just say "fetch this input, fetch that input, begin subroutine with vaguely suggestive name to label the following steps, compare the two inputs, fetch other data as needed, perform these side-effects, modify these variables, return to the beginning of some loop." Very verbose individual commands being operated on individual data structures.
So I tell you "make sure that if I press this button a hundred times you don't start the job a hundred times, only one job per project name should be running at any given time, but if I press the button while a job is ongoing you should still probably trust that I made a mistake when I first pushed the button and now I want to abort if possible and retry with the latest version" and because you don't have a rich vocabulary for that you're stuck programming something in a "redis assembly language" or whatever,
redis_job_flag = "job_id_for_" + project.name;
current_job = redis.get(redis_job_flag);
if (current_job) {
redis_complete_flag = "job_is_complete_" + current_job;
add_followup = redis.setnx(redis_complete_flag, "afterwards=run_again", 60*minutes);
if (!add_followup) {
followup = redis.get(redis_complete_flag);
if (followup == "afterwards=run_again") {
// someone else got there first
return;
} else {
// race condition, it completed before we could
// set the followup attempt, so we need to retry
// this method to restart it
return recurse();
}
} else {
// ok we can trust that they will redo what they just did
return;
}
}
// if we're here then we did not see any prior jobs.
my_id = generate_random_id()
try_to_setnx = redis.setnx(redis_job_flag, my_id, 60*minutes);
if (!try_to_setnx) {
// race condition, someone else got started before me.
// better still set a follow-up just in case.
return recurse();
}
// hooray! I am the unique one running the job.
redis_complete_flag = "job_is_complete_" + my_id;
looping = true
while (looping) {
doTheActualThing();
really_done = redis.setnx(redis_complete_flag, "complete", 60*minutes);
if (really_done) {
redis.unset(redis_job_flag);
looping = false;
} else {
// let someone else tell us to rerun again
redis.unset(redis_complete_flag);
// bug we ran into where someone made us rerun jobs
// for an hour, we reset the timeout for this here
redis.set(redis_job_flag, my_id, 60*minutes);
}
}
and then this requirement is restated for other buttons and that gets copy/pasted everywhere and other mutations get inserted in various places.
And I think what Ward would tell you is not "never write all those lines of code" but that this eventually needs a refactor so that it matches the business language,
repeatableDebounce("project_jobs", project.name, doTheActualThing);
// repeatableDebounce defined elsewhere,
// it is part of our "little world"
Basically, Ward wants you to start doing "aspect-oriented programming" because it turns out that is a felicitous perspective from which to view these requirements about needing to cache whether jobs are in-transit and if they are then to retry them and whatever else. Technical debt is the noise in the above implementation being pasted and tweaked across the entire codebase because we didn't have this perspective when we were starting.