A JavaScript Nightmare
blog.ignaciobrasca.com
blog.ignaciobrasca.com
A sane working environment would file a priority 0 bug and address it during working hours.
I'm not throwing a rock but oftentimes this sort of heroic behavior is self-inflected.
Also why is heroic debugging required on something that is not production? The article mentions it was a "quick demo". Then who gives a shit? There's time to investigate. And this would/should NEVER happen to production. The whole point of testing and staging/UAT environment is to catch this sort of problems. Again, a normal day at work in software engineering and not something to kill yourself living on amphetamine like a commando behind enemy lines in WW2.
It's really strange that they would invest all this time as if they are keen to understand the cause of why it happened, but then be satisfied when jiggling the handle solved it. The real cause ultimately is a process one (developer discipline) or a pipeline one (bad build/test/deployment processes). They got close to realizing it and then blamed the language instead of what appears to be bad practices happening on their dev team.
While I agree with everything you have said, and it's been my motto since I read, "I'm sure you're joking, Mr. Feynman." The point of the post was to point out (maybe I did it wrongly) that those things you have described are pretty impossible given the ecosystem we have nowadays in Javascript (as a whole!). Of course, you can try to theorize and try to get a hypothesis on things and then try to prove those, but it's just too hard to keep track of everything.
In addition, we are using lock files and good practices but it's just too much to handle for our small team (small startup) so we will plan to migrate onto a more stable platform.
It's really funny the heated arguments this post generated but I have been using Javascript for 12 years to these days, still this looks as a nightmare to me.
Tsdoing has a good video trying to install React from scratch, that's exactly how I felt during this debugging process
Another random story from this week: we have been using `nanoid` to generate IDs internally for an internal tool. Apparently, they did a breaking change release, and now you cannot use it anymore on commonJS env. See my point? where's the hypothesis we can make? It's hours and hours spent tinkering with code and other people's dependencies, and if, as you said, put a theory that theory won't match if you think how real systems work (the networking section on the blog post)
That's the thing, nothing you have asserted as impossible actually is as you'll see throughout this thread. I have been writing JavaScript/Typescript professionally since for about the same amount of time and I am not intimidated or challenged by my projects in the way you lament about.
That being said, if you've been at it 12 years and this is still your opinion, fair enough, try something else then.
If you are not intimidated by the complexities of this Babel tower, it's up to you. For me, this was just a personal catharsis in my personal blog that I wrote mostly for me (and I didn't share it here)
I believe there's no point on discussing this with particularly you since you have been commenting all around trying to just confirm your view instead of just comprehend that this is not sustainable for new developers (try to follow the conversations on packaging, lol)
It's simply too much burden just to maintain someone else mess. We will pay a high bill in the future for all of this.
I don't need to confirm my view at all. Advocating for one's view doesn't imply a lack of conviction, rather the opposite.
No need for a personal attack here. It's definitely sustainable for new developers who learn the stack. It has a learning curve but so does everything else. I've built applications in just about every stack there is (except for the more esoteric stuff like OCaml, Erlang, etc) and over those years complaints like those you raised are easily levied against any language or stack that is practically used in the industry. The common thread is that these sorts of complaints arise when there are more unknowns for a developer than knowns-- and as developers we are in the business of eliminating unknowns. When you encounter unknowns and your response is "this is why this tool isn't fit for use" instead of "let me turn unknowns into knowns", this is often the result.
> (try to follow the conversations on packaging, lol)
There isn't a packaging system out there without complications, but my posts regarding NPM's behavior points to how it actually already works to reduce or eliminate the version pinning failures you appear to have hit. Now if you are using very old versions of the tooling or if you do things like remove or ignore the version pinning tooling and how it should work, then yes the kind of problems you hit can happen. But that does not mean this problem isn't already solved, or at least the problem becomes an education one; which is why I'm out here trying to solve the education problem.
> It's simply too much burden just to maintain someone else mess. We will pay a high bill in the future for all of this.
That's, like, your opinion man. I've yet to find a development stack without it's costs, and once you are fluent in how to work in the ecosystem, Javascript's are not significantly higher than any of the others. And personally the cost/benefit of the Typescript ecosystem for me is quite rosy compared to the limitations of other stacks. I'm clearly not alone in that calculus.
Technically though, I have hardly met this dependency update problem with C++ simply because dependencies are rarely if ever updated. Android on the other hand was a nightmare. I had colleagues quitting in a rage and swearing to never again touch Android in their life. Probably half the effort was spent fixing crap that was broken by updated dependencies. Luckily I'm back to C++ now and hope never again will I have to touch such a crazy system. The whole 'update for the sake of update' seems to be a grave problem of the Android / Kotlin / Javascript universe. The mentality is like 'bread won't be sliced anymore, eggs won't hatch, sun won't rise if we don't absolutely have the latest bleeding edge alpha version of all libraries'.
Sure, I would also do long programming sessions on my own things every now and then, but those happen because I am in the flow and it feels like more work to come back into it the other day. But those are my own projects. I don't owe this to my employer. If they want that they could offer me double my current salary and even then I would insist on only doing it when it makes sense to me.
But I also live in Europe and my superiors want me to come in with a fresh mind, instead of burning myself for things that should be normal work.
That said, the main sense I get is that the team aren't so experienced with Javascript, and therefore working with it is full of surprises because they just aren't used to it. I found the line about bugs in C code being "your fault" particularly amusing: having occasionally had to touch C and C++ code, I would have said the complete opposite! Javascript, for me, is very predictable - if something goes wrong, then it's usually because I made a clear mistake. But with C, I end up trying to debug chains of Makefile and CMake problems that have no rhyme or reason to them.
In other words: as someone inexperienced in C, it is a chaotic, magical language and ecosystem to me, in exactly the same way that the author describes Javascript being to them.
More practically, it sounds like the team need some regression tests for this case that they can run next time they try and upgrade their dependencies. They also need to make sure that caching is working properly for their Javascript code - rerunning the CI multiple times on the same commit should always produce exactly the same dependency tree every time. Probably that means caching the right directory and using NPM CI to clear and rebuild node_modules from scratch every time.
More generally, the author complains about a lack of type checks. In this case, it sounds like the missing method was somewhere in a dependency, so this may not have helped, but: Typescript! It exists, it adds types, it makes a lot of things easier. Easy mode Typescript is to just start writing `.ts` files and use the `tsx` package to automatically compile and execute everything - there are other approaches that they could explore, but this is probably the simplest.
> The next message I sent: “We found it!”
May be I misunderstand something, but from the post it sounds that they don't even commit their package-lock.json. They're using node like its 2012.
To be clear, I agree that the Javascript ecosystem isn't perfect, particularly when it comes to supply chain issues. But in a lot of ways it's more secure than Python's typical dependency setup. By default, NPM uses a lock file, which enforces that specific versions of existing packages are used unless explicitly updated. This means that you always know which versions of which dependencies you're currently using, it allows you to upgrade different parts of your dependency tree independently, it ensures that when you remove a dependency, all of its transitive dependencies also get removed, and it allows for different dependencies for production and development environments, and for different portions of a given project.
Doing all - or even part - of that in Python is difficult. There's no built-in way, and so people tend to adopt third-party tools or roll their own scripts. The roll-your-own process, in my experience, almost inevitably leads to some sort of failure, usually a certain updating unexpectedly and breaking everything, and then not being able to roll back to a previous version of that dependency. The third party tools (Poetry, Pip-Tools, etc) work better, but are often difficult to integrate more widely into the Python ecosystem.
I don't want to make this about Python vs Javascript - both are tools that make different tradeoffs and can be valuable in different ways. The point here is that, as software developers, we often end up with what are essentially superstitions about other programming languages - we describe things we don't know or don't understand as "magical" or "scary", because that's easier than admitting we're just not that well informed. And while it's okay not to be informed about everything, in situations like this, it can lead to bad outcomes.
In the article, the author ends up blaming the situation on the magic and weirdness of Javascript, but it sounds like the issue had more to do with them not understanding how best to do package management in Javascript.
And if you want reproducible dependencies, use npm ci not npm install, or use yarn. Best combined with replicated self hosted npm repository.
Feels more like an oversight from the developers rather than a JavaScript problem.
When you want to upgrade, only then do you run all regression tests and do so.
However, "npm install" also adheres to your package-lock.json, and does not update things on you.
npm ci is just npm install with a few tweaks that make it better to use for CI and builds, see here: https://docs.npmjs.com/cli/v8/commands/npm-ci
What do you mean by this? You can specify dependency versions using "^" which means "I accept new minor and patch versions" or "~" which says "I accept new patch versions", or a bare version to say I want an exact version; but none of those have anything to do with a bare "npm install" command. They are relevant when you install a new package, with "npm install <package>". That is a totally different operation, and it absolutely can update versions when needed.
However, when you just do "npm install" it does not update your package-lock.json to the latest version that matches your semver requirements in package.json. That will only happen if you install a new package that needs it or you use `npm update` or some other command.
This is easy to test but it's also intuitively correct- you don't expect to run `npm install` and have an uncommitted change to `package-lock.json` appear, and that does not happen in normal usage.
It is kind of unfortunate that "npm install" and "npm install <package>" use the same verb of "install" for two different things- other package managers don't make this mistake and it avoids the sort of misconception we're running into here.
It makes sense that they behave the same here right? What is the point of a package lock if just installing packages on a new copy of a codebase updates the dependencies?
Package locks aren't just about deploying: As a developer, I need to be assured that the code I'm running is the same as my coworker.
Node 16 shipped with NPM 8, and by then this was not the behavior any more. I don't remember exactly when, but I believe it might've been in NPM 7 when this changed. Node 15 would have been the first release to ship with at least NPM 7 [2]
EDIT: Node 16.0.0 shipped with NPM 7.8.0
https://github.com/nodejs/node/blob/main/doc/changelogs/CHAN...
EDIT 2: A StackOverflow confirming NPM 5 behaved the way you remember: https://stackoverflow.com/questions/45022048/why-does-npm-in...
EDIT 3: And, as that SO documents, 5.4.2 changed the behavior to prevent this, here's the GitHub issue: https://github.com/npm/npm/issues/17979#issuecomment-3327012...
Honestly no guarantees it worked as intended right at that release, but the intention is clear.
EDIT 4: I realize that the following may present retort but it matches my mental model perfectly:
> If you manually edit your package.json to have different ranges and run npm i and those ranges aren't compatible with your package-lock.json then the latter will be updated with version that are compatible with your package.json. Further runs of npm i will be as with 2 above.
This is what I've mentioned elsewhere: You should not update package.json's versions and deploy it and expect that versions won't change when NPM install runs. In fact, npm ci will just fail in this situation, which is a Good Thing (TM), and is documented as such.
LAST EDIT SRSLY:
It is conceivable, especially given OPs other red flags, that perhaps they were on an older Node/NPM like Node 14.
That's another practice that I regularly do, pin NPM itself in deploy scripts. While I was using Node 14, I was using NPM 8, because you can just "npm i npm@8 -g" to upgrade your NPM regardless of Node version.
[1] https://blog.npmjs.org/post/171556855892/introducing-npm-ci-...
- Use Typescript.
- Practice version locking on your dependencies and make sure it works correctly throughout your processes
- Do not commit vendored dependencies
- Do not commit compiled JavaScript
- Use a CI process for deployment which runs your tests
- Have tests
- Prefer (or require) types for your dependencies, or dependencies written in Typescript
- Prefer dependencies that ship the types in package (and not via DefinitelyTyped)
Some additional thoughts, they act like running out of memory is something they need to actively handle in the codebase-- it's not in this case. It shows a lack of familiarity with how JavaScript runtimes work which hints at a team which is very inexperienced with the platform. JavaScript runtimes enter GC lockup and ultimately crash if there is no memory, so this is a logging problem, not an error handling problem. There is no mechanism for the runtime to inform the app that an allocation failed.
Dependency silent upgrades and dependency hell and bloat are not unique to javascript, but it feels like the js community has a perverse fetish for fragile build systems and dependencies.
Ofcourse, the community is not one organised entity and this is merely the result of multiple individual decisions taken independently, but still. I can not help but empathisewith the author despite seeing that they aren't locking deps, not doing reproducible and rollback-capable deployments via docker or artifact tarballs, don't seem to have robust ci, or good work life balance..
Is it elitist to remark "haha, don't you know you should smerp the blergh?" Is it not atleast somewhat crazy that we normalise rube-goldberg-esque contraptions?
I don't even know what I'm getting at, at this point honestly. Js gives me very mixed feelings. I love the accessibility amd universally available execution runtime (browser), and understand why it hassome of the warts it does, but it is also honestly a scary ecosystem with hyper fragile tooling. I've personally burnt multiple full days on webpack config tweaks and it all feels so unnecessary and pointless.
JS almost gives me existential crisis at times.
Packages built by 3rd parties are the Wild West, and unless they’re a full-on company of dedicated, experienced engineers, they’re gonna break semver best practices. They’ll also design bad APIs for interacting with their packages sometimes, and there will be a total lack of consistency across the ecosystem, which is what we have currently.
If I had more free time, building a stdlib for JS would be high on the list of “help the world” activities.
It sounds to me they rolled in the dependency update with their features, then had a massive surface area to cover when it flaked out.
This is true in many other languages, not just Javascript. This isn't a "Javascript nightmare", this is a dependency nightmare.
I use Kotlin, both on the JVM and in the browser (via kotlin-js). Dependencies are equally complex as in javascript but it's much easier to find out if stuff breaks or not. I update often to minimize the amount of changes I have to deal with. This may sound counter intuitive but integration testing small deltas is exponentially easier and less risky than big deltas.
My attitude to having big updates lurking is that they represent technical debt. Just because you don't know you have a problem because you haven't updated yet doesn't mean you don't have a problem. By not updating, you just increase your ignorance of technical debt. It might be piling up without you realizing. And you might be making things worse by working against obsolete APIs.
Upgrading libraries often means I typically only have to deal with a handful of minor updates for libraries that are mostly unproblematic. And if stuff breaks, it's easier to pin point the source of the breakage. And I find out as early as possible about problems I need to deal with. If I don't know about it, I can't deal with it either. Sometimes downgrading a library temporarily is needed. But then you document why and stay on top of issue trackers and upstream development to see when you can upgrade or you start looking for alternatives.
Occasionally, some API breaking changes happen and then I fix it (because the compiler tells me stuff is not compiling). If I don't update for a year, I probably have a lot of breakage on my hands and a lot of work fixing things. Much harder and much more time consuming. And risky. All I know then is that I probably have a lot of problems (i.e. technical debt) on my hands that I don't know about.
Maybe you've never heard of "writing tests"? If you update a dependency, and the dependency has a breaking change, your tests will fail. It's really that simple, and this should be the same in any language, not just Javascript.
That being said, there isn't too much about Kotlin I miss when I write Typescript in strict mode.
> Unlike typescript, it does not really have a non strict mode (except for the dynamic keyword so you can deal with some js libraries and its weird type system).
Why is the existence of non-strict mode a concern for you if you are using strict mode? After all, whether you are using Typescript or kotlin-js, you are likely to have JS dependencies, and those might not be written with Typescript strict mode. I guess you could just opt not to use any "native" (Javascript) dependencies and just do it all in Kotlin JS, but beyond a certain point that's not very practical.
I do indeed do a lot in Kotlin-js rather than using javascript libraries. We make a few exceptions for libraries like map-libre for which we have a very thin hand crafted Kotlin facade. Things like react we simply don't use at all (we use a pure kotlin framework called fritz2). We actually use a lot of Kotlin multi platform libraries as well. And even some wasm libraries (e.g. sqlite). Most of the foreign code dependencies in our codebase are not javascript libaries.
> @smithy/is-array-buffer@2.1.1
Are these direct deps? Because just a cursory look raises all kinds of red flags. There's a disclaimer for the former warning about it being in developer preview, and the rest of the README has a bunch of notes about edge cases.
I get that the aws-sdk uses them, but if it works for other people it should be fine. Having these as direct deps and also calling in complex libs like the aws-sdk honestly seems like a recipe for disaster.
Well, here is your problem.
Yes?
Also, after coming from real compiled languages I find TypeScript ecosystem not to be robust at all. Change in one file sometimes does not result in compile time error in another file where changed type is used. Yes, it's not an issue with TypeScript itself but with its ecosystem, transpilers and compilers, but that's not something I experienced ever with C# or Java.
> Change in one file sometimes does not result in compile time error in another file where changed type is used.
Sounds to me like you are missing types or the types you have are incomplete.
When you do things correctly, this is not an issue. Part of doing things correctly is preferring (and in some cases requiring) that your dependencies either be written in Typescript or at least ship the types directly in the NPM package, per best practice.
I'm not talking about npm, I experienced these issues in my own code. It's mostly related to ts transpilers. They're fast, they're used in build systems which should be latest and greatest (e.g. Vite), but they're not robust, they don't always catch type changes in dependent files. And now I can't just change a type and rely on compiler to check ALL occurrences in entire code base. Because it's not really a compiler but a transpiler and it misses some things. Also, tsc doesn't seem to handle jsx/tsx, is there a way to perform real type script compilation with full type checks on tsx files?
Most systems built on Vite though do check across files, so likely that's just a bug in the build system you were using. A bug that is likely fixed.
One thing you can be assured of is that when you actually build your app (hopefully using CI), it will find those problems, because you are not relying on hot reloading to do it.
On the backend side it's far less common to use a build server model, and personally not very warranted -- building a frontend app can take a lot of time and the feedback cycle needs to be short for a dev to be effective. On the backend, a small amount of time rebuilding the app before it is rerun is not nearly as costly. If your Typescript codebase is so large that the feedback cycle of rebuilding is too long, well you have to take the tradeoff, but it's a rare problem.
Understand that the only way a "dev server" style backend build process is saving you time is by not checking the types during recompilation. It wouldn't be surprising that it would miss typing changes.
The point still stands.
> getting huge “silver bullet” sentiment from your replies
???
So all the dependencies would need typescript?
The article is not very concrete in what was the actual bug
I'm not sure where that quote comes from, but these days in terms of TS vs JS it's 100% "not using untyped JavaScript for any serious product".
And then to prevent this, you just lock the dependencies via a lockfile -- a well documented solution.
This is just another amateur "javascript bad" post.
Unless of course your handler results in dying for science, then it's fine (ie for sending to an observability service first)
Not sure why I can't downvote articles, ah well.
I will say this though: it is infuriating that package managers (every single one I've used, not just NPM) don't default to pinning exact dependency versions, and instead let it "automatically" update minor versions.
However, if you need to integrate a new dependency it might require a later version of a package you already installed. NPM is clunky at times but most of the times it works quite well — despite all the micropackaging madness.
Note that actually using package-lock.json to resolve dependency versions is not actually a difference between npm install and npm ci
The real benefit is that npm ci will refuse to work if package.json and package-lock.json disagree, which protects against changing package.json without running NPM install during development.
Otherwise, it changes behavour, and has the potential to break your software.
In practice, almost all changes should be major version bumps.
Anyway that's besides the point. Let me control when version changes happen!
NPM already gives you a way to control when version changes happen, it's called package-lock.json, and it's used for all NPM installs (both in development and when deploying). Just don't listen to the bootcampers on StackOverflow telling you to delete it whenever anything looks weird.
If you delete package-lock.json, assume every dependency version has changed and regress accordingly. The same is true of all reasonable version-locking package managers.
Ha! Think again. On of the more annoying problems I had to analyze was when a new PHP version added a function that already existed in my application (this was before namespaces). The runtime did not like that.
And our docs would mention both the old and new way so if you were searching the Internet for help, you wouldn't get confused when you saw wildly completely examples.
It's all about craftmanship.
If you listen to very bad advice though, half of the Q&A answers on the Internet suggest all problems can be solved by deleting package-lock.json, to be fair.