On Modern Web Applications Stability

When I started doing webdev 8 years ago you shipped a recentish copy of jQuery, maybe jQuery UI if you were feeling fancy and that contained pretty much every library function you needed. Now we NPM install thousands upon thousands of modules to get by.

These days, browsers have tons of library features baked into them - but with APIs so obtuse and difficult to use you have to install another library to make them usable.

Last week at work I had a 2 hour meeting where we tried to work out how to ship the 2MB of JS our website apparently needs to clients without Google Lighthouse penalising us.

I don't know what to do with any of these thoughts but they often make me want to quit and take up a profession that doesn't involve computers like blacksmithing :/

About one week ago I woke up to my great uncle knocking on my RV (we live in his driveway temporarily) and he said, the fastest moving fire in CA history is moving toward Paradise again and we need to get ready to evacuate. We had no electricity and so updates were bad. Later he updated us after driving to the top of the hill that the fire was 40 miles away. Later that day I found out he was wrong, it was 10 miles away :P. The sky was red, the world had stopped, everything was hopeless.

I went to gas up the RV, hundreds of people doing the same, all scrambling trying to avoid a repeat of mistakes from 2 years ago. Everything is nuts, no one is calm.

Except the world didn't stop. Mail was delivered in Magalia and Paradise that very day. UPS and Fedex dropped off packages in both cities that very day. The world is actually a lot more robust that we give it credit. Sure, maybe we are constantly 2 minutes to midnight. But maybe, just maybe this modern society can handle pandemics. I know the literature is saying the exact opposite. And maybe in a year we're all going to be starving to death. But if that's the case, then the least of my worries is how many dependencies I have. If anything, if I have more, perhaps we're strengthening our society just a little by paying 5 SAAS companies to keep mine running.

Just tying it back to my fire experience, I depend on daily, so many freaking companies and people, ACTUALLY doing stuff that day, that I don't care that my code is resting on the backs of 4000 developers I haven't met, that haven't touched 90% of that code in 3-4 years anyway.

Not saying I wouldn't enjoy a completely different society, but that's a different essay.

For any reasonably successful web application there comes the day when a developer has to investigate the cause of a bug who wasn't there when the application was "young and agile". They will scratch their head above old libraries, out-of-fashion frameworks, tools, and languages, and finally, more often than not, argue that the whole system needs to be reimplemented in X.

However it is a little short on solutions. The truth is that I don't want to write my own PayPal API when there is one that PayPal produce and are likely to keep more up-to-date than me. Writing my own means that I only rely on my team but it is a big overhead for any reasonable size company that use dependencies from possibly 20+ major libraries and the hundreds of lower-level dependencies.

What would be cool would be a webpack style dependency manager that can remove things you don't use and then only show you updates for the bits you do.

Basically entire human civilization is based on collaboration, learning from and relying on each other (among other things). The process kickstarted with developments in agriculture, freeing up Human Resources to work on other things. We’ve been perfecting this for a few thousand years, of which we’ve been automating things using software for only 50 years (being generous here). Give it some time - with the current rate of development we may not need another thousand years to improve software development to near perfection as well ;-)

(Or more pessimistically: there’s still a lot of hunger in this world so even after a few thousand years we’ve not even perfected the most basic of human needs - doesn’t bode well for something as fancy as the typical glorified beans counter that is most enterprise software...)

Disclaimer: This video is typically used as a pro free markets argument to illustrate the level of cooperation required to produce even the simplest things, arguing that such collaboration is only possible by free actors in a free market - that’s not the point here but still interesting tangent to think about in this context.

I'm not arguing that package bloat is a good thing, but I do think it's an acceptable trade-off for the speed and functionality improvements.

If long-term stability is a very high priority for your project, you should look at using the right tools for that job. A language like Clojure that actively seeks to avoid "churn" while operating on top of common runtimes is a good choice for things like that. There's no need to compromise on speed of development, either :)

For example, if I want my users to be able to scan a generated qr code, I am not going to write a qr code generator myself since that is quite a lot of work. I will use a trusted library to do that for me.

In this way, the library will have had a lot of eyes, patches all the time fixing bugs and so on. If I would try and make a QR code generator, I would most likely make a lot of mistakes and bugs that more seasoned people that have experience in making such functionality already have experienced and fixed.

If I stopped writing code for the company I work for today, the application would work just the same in one year, in two years etc. I don't get why a desktop app would be more stable than a web app in that sense. I would argue a web app is more stable since we don't really break backward-compatibility in a browser in the same way people do in desktop operating systems.

Sure there is something to be said to not use a big ass framework for some small app. I think frameworks usually brings more pain than it's worth in the long run. But having dependencies in modern applications is a must, even in desktop applications because people expect and demands it. If you would not use dependencies, your customers would choose your competitors most likely.

"As if source code rusted. The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive."

By that token, blindly updating your dependencies sounds more like the problem and so does writing your new code in place of their old. Security patches may force your hand but apart from that, if you want stability just be deliberate about upgrading and it will be rare for old dependencies to suddenly generate new bugs.

As far I see it, the right way to think about good, reliable code is not as a compact, resilient thing like a baseball but like a matchstick sculpture glued together with maple syrup. It is both inherently very fragile and very unlikely to spontaneously break if good decisions went into the design of its supporting structures and because it's mounted to the sturdy, climate controlled plinth of the silicon chip.

In the article, these supporting structures such as compilers, OS and firmware are mentioned but if we don't trust these then we simply should all stop trusting computers. Most of us believe that this broad societal dependency has been a good tradeoff bringing us an amount of good which is so hard to outweigh even by an imagined disaster.

The same goes for most of the dependencies in the average ruby project (don't know what to say about JS). They are battle-hardened and bring value that companies would pay massive sums for were they not free.

There are of course the odd suspect, unnecessary dependencies in most projects. Usually they're for trivial uses and easy to remove. When you discover these few they do not indicate that the rest of your package list was a mistake. The fact that 90% of the problems you have come from 5% of the dependencies (made up statistic) should tell you that your project will be dead before the other 95% let you down.