Speeding up the JavaScript ecosystem part 2 – Module Resolution
marvinh.dev
marvinh.dev
This isn't just to "save a little bit of typing", there is a significant amount of mental overhead eliminated by not having to recall and reason about the relative path relationship between each module for every single import, it also prevents the issue where moving around files or refactoring your directory structure breaks your application, forcing you to do potentially dozens of import path updates depending on what got moved around.
Originally, I was hoping that "fast module resolution of huge projects" is something that Deno would have solved from day 1 when they said "meh we don't need npm" and had the opportunity to clean-slate the design. Granted, they do support npm now, but personally I think they only had to back-peddle because their non-npm approach didn't have any huge advantages (i.e. perf wise) for most users to bother with.
Like if Deno had fixed this "30% module resolution overhead for large test suites / app startups" when it first came out, I would have gotten our app on to it immediately.
IANAE / I haven' tried it yet, but I believe bun's `bun bun` command is the best (only?) innovation trying to tackle this problem:
https://github.com/oven-sh/bun#bun-bun
I really should it try, my hope is that it's the "I am _immediately_ moving to bun" carrot that Deno never delivered.
Thanks for the feedback. The reason I wrote this article is because we _can_ do something about it. Through avoiding throwing lots of wasteful error objects and adding a little bit of caching, the time it took to lint the project became 30% faster. Those changes were applied locally to a couple of popular third party tools for module resolution in node_modules. That's how the 30% speedup was achieved.
That said if module resolution wasn't as complex in node to begin with, the speedup would surely be a little greater. I'm hoping that this post sparks a bit of discussion on that and some node contributors already voiced interest on twitter to think more about that.
Out of curiosity: did anyone end up filing issues or PRs against these tools as a result of either of your articles?
The PRs for the previous article were all merged. I originally wanted to do the same for this one, but I'm not sure if I have the time to fix all of them. Updating to a newer version of `resolve` already addresses the most notable issue with throwing more errors than necessary, but many parts of the ecosystem still use an old version.
Manged to land this PR in another package though https://github.com/import-js/eslint-import-resolver-typescri... . Performance there could be easily improved further there.
Another PR to an eslint plugin was unfortunately rejected as it broke node 4 support https://github.com/import-js/eslint-plugin-import/pull/2654 .
The other popular package that's used for module resolution is `enhanced-resolve` by the webpack folks and they expect the consumer to deal with passing the appropriate options. So there isn't really a single place to fix this.