Forcing Functions in Software Development
coderefinery.wordpress.com
coderefinery.wordpress.com
I wrote the code, and got it working in about 2 months. It did everything in the spec, and the customer (Russ) loved what we had, but it turned out (of course) there were many things missed in the spec.
A week or two later, we worked out a deal for all of their power plants, provided I would work with Russ to make sure it did everything they wanted... it would take about a year, in the end. Russ taught me everything I needed to know about users and how they think, in a direct and very effective manner.
He was the assistant plant manager, so he would pull a random person into the office, and say to them "I know you're not a computer person, I want to you to do X,Y,Z... but don't worry... if anything goes wrong, it's not your fault... it's Mike's fault (Russ points at Me). He then tells me to just watch... It took exactly 1 minute to learn the first lesson... there should always be "Press F1 for Help" somewhere on the screen.
It was a very instructive and productive year. I've carried those lessons from his forcing function with me for decades. I love telling the story, thanks for listening.
Well, details of Postgres have leaked into my database queries and schemas, but it was a conscious decision to use Postgres and its features. Sure, it seems nice to be able to swap one database for another, but you lose out on a lot of what a database can do if you stick strictly for the lowest common denominator of features. I use Postgres partly because of its feature set, so I am going to use these features. This does mean that its unlikely I will ever run my software against a different SQL database, but I'm ok with this.
I guess an important note is that you should be aware of it and it should be a conscious decision, rather than something that crept in over time.
But I suppose that's a tangent and not the articles point. Forcing Functions is about unearthing the brittleness that has crept into a codebase over time and I absolutely agree that is a helpful and worthwhile exercise. I've seen plenty of codebases that were meant to be database agnostic, but porting them was still not painless.
How often do people really migrate to a different database? Even when doing so, migrating the existing data always is a lot more tedious than migrating the code, in my experience.
The same goes for platform code, really. If you are advertising your platform to work with databases X, Y and Z, then you really need to be testing against those and not just as a Forcing Function to see if any brittleness crept in, but as a core part of ensuring your platform does as advertised.
Yes, many libraries need, or are enhanced by, persistence, whether that's a double-entry accounting module, a headless cms, or an e-commerce engine.
> you really need to be testing against those
Library authors need tools and guidance, not arbitrary constraints and high-watermark QA demands. Most of us have one or two DBs that we work with day-to-day but still want libraries we contribute to be broadly portable across any of the backends the framework they plug into supports. e.g. I can't maintain a test suite vs Oracle when I don't license it, but I still want to know that It'll Just Work if someone uses it in such combination, or that it's at least close enough they'll not have trouble making it work.
More broadly I think the myth of testing every supported combination leads to a brittle mindset of saying "not supported, won't help", especially when systematised in a commercial environment, and to me it's the antithesis of good engineering.
In the context of the article, for "Forcing Functions", I absolutely agree that switching databases is a useful way to find weakness in your solution.
Well, I have been part of a migration from MySQL to MariaDB, which was a lot more effort than one would expect given that they're meant to be more or less the same thing. It was a ton of effort and the abstracted ORM logic didn't actually help with this.
So if it doesn't really help that much for a simple case like that, then it doesn't seem like there's much point in my opinion, as porting is going to be work either way (in a non-trivial application with non-trivial data access patterns, at least).
It wasn't a big deal over all, but it wasn't simple plug'n'play either and required some porting work.
On the other side, I worked on a web app that supported multiple db vendors, we did the classic DAO pattern which worked well. You still get to use custom SQL for each database if you need to.
We tried an ORM at one point which worked out well. It was the same web app and we moved moved some DAO code to the Java Persistence API. We could then build the data access code and include it into our desktop (Mac Windows) and plug it in to a local DB (Derby).
In that case, once JPA was working, the pluggable database was allowing us to save on development costs.
I tend to agree, as long as the database you’re using is of the free and open source type so it’s a low-risk dependency.
The “pluggability principle” holds more strongly for higher-risk dependencies, IMHO. For example, if you’re working in an online environment and you can’t readily integrate a new payment processor or messaging service or whatever, your existing dependency starts to look like a single point of failure that is not under your control. Some careful abstraction of the essential features and avoiding dependencies on the peculiarities of any specific platform can be a valuable safety blanket if your external dependency catches fire one day.
Basically, it's impossible to write a contract to cover all situations, so eventually any two parties will need to renegotiate. If one has leverage over the other, they can do stuff like raise prices or simply refuse service. One factor affecting that leverage is "asset specificity": given an asset that's covered by the contract, is it reusable in different contexts or is it too closely adapted to one purpose?
The literal textbook example is a steel foundry and a railroad. The steel company needs a spur line to its foundry to receive raw materials and send out finished steel. The railroad company can run a spur from its main line to the foundry.
In this case the asset (the spur line) is extremely specific. Without the spur line, the foundry is worthless. Without the foundry, the spur line is worthless. The balance of leverage then comes down to relative costs and gains. The railroad will hold the upper hand, because closing a spur line isn't existential, but it is for the steel foundry.
In the software context it shows up as a "contract" between your software and a database. You can use an ORM to try to insulate yourself, reducing asset specificity, but that decision has costs (eg, specific DB features you must forego). You can choose to increase asset specificity by adapting to only one DB and fully utilising its strengths, but you may face a future hold-up.
Note from the train example that hold-ups needn't be "pay me more". They can also be "I'm walking away". So a hold-up from Oracle will be "pay me more", whereas a hold-up from an obscure database with one developer could be that they simply quit, die or otherwise become incapable of working.
If they already have an Oracle license, or a MS-SQL DBA around, you either say goodbye to Postgres or give the contract to your competitor.
Presumably the idea is based on an analogy to code portability: it can be good to ensure your C++ code compiles fine with multiple different compilers. Really though, it's more akin to writing code that compiles as both C# and Java; clearly madness.
Adding support for a different database doesn't mean restricting yourself to the lowest common denominator. It means using different techniques, more appropriate for the different database, that may optimize other parts of your data access and modification path, while pessimizing stuff your current database does.
More importantly, it means extracting hard database dependencies like raw SQL or custom ORM fiddling from your business logic and entities, and pushing them behind a module or service boundary.
Raw leverage of the database, if it's dispersed throughout your application, will limit your ability to change your schema (e.g. denormalize an attribute, split or join tables, convert a parent-child relationship to embedded JSON or vice versa) and address performance problems as you scale up. It'll also stop you having a single point of data access where you can partition or duplicate your data into different stores with different capabilities more suited to their access and modification patterns. These kinds of things become really important when the database becomes a bottleneck in your system.
If so, then, sure , I agree with you. Not all reliance on a target database is actual features that don't have an easy or direct way to port.
More that being forced to separate church and state, forced to keep business logic separate from database manipulation, and forced to go through an abstraction interface, then lets you leverage the interface later if and when the database becomes a bottleneck.
It's a devil's advocate position. I actually think it's unrealistic and probably not worth it at the time. The forcing function is what's useful, not the database portability per se. And whether it's actually useful depends on being so successful that you have so much data that your regular RDBMS can't cope with some access or update patterns. That's a hefty bet early in anything, and probably doesn't make business sense, until it does, then you wish you did things differently.
(Not coincidentally, it's a position we're in at my workplace; the database has hit its limits and we're needing to use hybrid approaches to hit latency NFRs.)
So my app can make calls which will persist data to a SQL database, or I can swap that layer out with another that persists to NoSql storage in the cloud. Possible because the persistence layer exposes an API or interface that is agnostic to the actual implementation of the layer
???
While the list of techniques here look like excellent ways to uncover unknown unknowns, be sure that it's actually valuable for you to resolve these issues.
In almost all cases the goal isn't to create perfect software, it's to create effective software - and sometimes the ROI on these unusual cases doesn't stack up (or doesn't stack up _yet_).
Very good advice. This was obviously never done in my org, leading to newcomers (me included) wasting weeks to get started on some projects. Once it was fixed, a newcomer can start working in an hour.
Though I once inherited a large enterprise code base that I had to study and build a dev environment for pretty much all by myself. I had maybe 2 calls with the original team and a couple of emails but was mostly by myself just figuring it out. It was a pretty incredible experience and taught me a tonne about how the system worked and was put together. This helped immensely when we spun up a team to work on it. So, I get where this article is coming from due to that experience, but didn't think to do this kind of stuff on purpose.
It's an extreme intolerance to imperfect circumstances: a preference for nothing at all over compromise.
The issue is that tech attracts the mathemetically-minded who reasons from universal principles. Rather than the empirically-minded who start with cases, and abduce to provisional principles from those.
To a aximoatic mind: when a universal principle is violated, the situation is declared Bad.
To the case-base mind: when a tolerable situation seems to violate a principle, declare the principle Inapplicable.
Of course both types of thought are helpful in different contexts, I suspect 'the management of one's life, day to day' however, should be a matter of case-base reasoning to rough principle.
> Everything is a rush and quality is required but never budgeted for.
It doesn't really sound like this is a situation where the principle is wrong, it sounds like this is proving the principle correct: you get what you pay for.
That trading off thought process isn't the same as the ACCEPT|REJECT process of the axiomatic mind.
I think a person who says "quit" is really saying that the very question of trading off heuristics of value isn't applicable.
It's a bit like Poisonous|Edible, or Gold|NotGold. There's nothing to be traded. A Copper apple is Poisonous and NotGold. No two ways about it.
That reasoning only works when the concepts (Gold, Edible, etc.) are natural kinds -- or otherwise disjoint and universal classifiers.
In life, situations fall both into the ACCEPT and REJECT categories, into both GOOD and BAD, into both VALUABLE and WORTHLESS. These concepts are heuristic ones, and not disjoint & universal.
The attempt to apply this "disjoint, axiomatic, ..." reasoning to life is a recipe for catastrophe.
EDIT: my point about principles vs. cases, is that i take: def. heuristic "a resemblance amongst cases"; and def.., principle "a universal rule which disjointly classifies cases"
..ie., a slightly more extreme meaning to "principle" than is in general use
I sort of agree and am just playing devil's advocate, but I also think everything of 'there's better things out there you know' has probably already come up thread just before the person complained of their situation not being as good - they know better's out there at that point.
It's a bit of an argument for "less haste, more speed".
As an example, I've always found with CI (continuous integration) servers that setting them up on day 1 of a project takes almost no time, but trying to set them up 3 months (or later) into the project seems to require a lot of time. Once a CI server is set up, it invariably improves both quality and productivity significantly, they start yielding dividends on their time investment very quickly.
If management claims they can't afford the relatively small amount of time required to set up a CI server, then I would argue that the lack of time only strengthens the need to have it done sooner to enable the project to move faster.
For onboarding documentation, finding random time may be hard, but it's almost free if it's done by a new member as they join a team. As they get their environment up and running, they just need to document the steps as they went along. Make sure it's committed to the same source control repository, readme.md seems to work well for this. It's fine if it's initially very simple, just an unformatted list of steps in plain text is a great start.
If someone is adding new technology stacks which would would affect the onboarding document, they should quickly add it at that moment, while possibly improving it a little by adding a little formatting. Future new team members should also be encouraged to improve the documentation based on their onboarding experience. Over time the document becomes quite refined and easy to keep upto date.
That doesn't address all their points, but it's a start and I hope it's helpfull.
To borrow a line, if you think quality is expensive, try cutting corners.
https://smile.amazon.com/Accelerate-Software-Performing-Tech...
If there's a bug slipping through, someone will run into it, report it, and it'll get fixed. If nobody runs into it, the bug doesn't cost anything.
This is the most economic way to go about it, which is the reason why pretty much all successful software is kind of buggy. We all like to complain about it, but then we don't want to wait an extra year for the next version either.
At the other end of the spectrum, if you need really reliable software, the solution is not to eliminate all the bugs, that's impossible. Even with perfect software, hardware can fail, bits can flip the wrong way. The solution is to make sure that errors can't bring down the airplane.
Also, the bug that would be discoverable locally but not in production is a bit of a strange animal. It does happen for cases where the developer has a better idea than the user what should be happening so the user will not even notice the bug. That sounds more like features that have been specified in too much detail than true bugs, though. More commonly everything that can go wrong locally will go wrong in production sooner or later. And in production you will, on top of that, hit all of the bugs that occur once every month and that only occur if the order of inputs is a bit strange while the database is under some load. If you have not done all possible debug/test work locally the application will hit production when there still is a lot of debug work to do and there will probably be so many problems that they even start interacting with one another and produce some really 'interesting' failures.
Why do you think the customers are going to report bugs? I can't say that I've ever attempted to report a bug on an iPhone app, Windows, etc. It's like yelling at the wind.
Correct.
> Why would the customer report a bug instead of switching to a less buggy product?
All products are buggy to varying degrees. As a user, you can't easily tell if another product is "less buggy". Would you risk investing time into figuring out another program that may turn out just as buggy? No. You most likely move on with your life.
Once users are invested into your product, it takes a lot to make them switch. Every single piece of software that I use regularly is either extremely simple or somewhat buggy. I haven't switched once because of it. It's a nuisance, but not a dealbreaker.
That's not to say that I like this situation, of course I would prefer software that doesn't have bugs and glitches, but I also prefer software that exists today, not tomorrow. The market has spoken: worse is better.
> Why do you think the customers are going to report bugs?
If it's an important bug, somebody will most likely report it. If it's not an important bug, not having it reported is most likely not important either.
> I can't say that I've ever attempted to report a bug on an iPhone app, Windows, etc. It's like yelling at the wind.
So, did you switch to Android or Linux/MacOS then? Is the grass really greener on the other side?
Is your motto "When life gives you lemons, make a market for lemons?"
The highest-quality software can not win in the software market. This is evident from the software that is out there owning the market.
Quality is a trade-off, if you spend too many resources on it, you can not compete. Catching those last few bugs takes exponentially more effort.
Moreover, there are snake oil salesmen at every corner, telling you that if only you adopted some methodology, your defect rate would plummet. It's easy to get lost in that, not actually delivering a product.