ES modules: A cartoon deep-dive (2018) | Hacker News Reader

ES modules: A cartoon deep-dive (2018)

hacks.mozilla.org

102 pointsluu36 comments

If anybody has tried ES modules in the past five years and had to turn back due to incompatibility / lack of library support / javascript is a minefield, I'd urge you to try again. I managed to migrate our Typescript/Node codebase over to ESM about a month ago, with only a few hiccups. It's nice to be able to use the latest versions of all of sindresorhus's packages again.

If you have one build pipeline, I would recommend it. But it's still been a nightmare for me. Key issues are some packages are 5+ yrs old and not maintained, and we have a very complicated setup, one common repo shared by a Next.js app and custom Koa app. Every tool you work with, like Jest, needs its own workarounds.

I have noticed more and more libs are exporting ESM, but lord knows when we can stop adding special compiler rules.

Kinda niche, I wrote Bazel rules (https://github.com/rivethealth/rules_javascript), and with a build setting `--//javascript:module`.

Jest tests automatically use the CJS version. Webpack builds use the ESM version. (And all my stuff is in TypeScript, so it needs a build step anyway.)

Jest on jsdom is just terrible though. It's a fake browser environment that acts like no browser that your users use, so the tests are far less meaningful that if you used a real browser.

If it matters that much that your tests are in a real browser environment what you are writing are likely integration tests, rather than unit tests. jsdom is great for unit tests that need to test a bare minimum of side effects.

I know the distinction between unit tests and integration tests doesn't matter to some, but I still see a huge usefulness in distinguishing because unit tests should run in the "inner loop" every time a developer is touching code (so must be fast to avoid sapping productivity) and integration tests can be delayed until the "outer loop" (CI processes and UAT processes) so are allowed to be slower. Booting up a "real" browser is definitely on my slow things list and not something I think belongs in unit tests.

Well, for browser tests I use Selenium.

(Tho I still use jest as the test runner.)

dependencies that we need which are still CJS-only are what kill our ability to switch completely to ESM :(

You can use Parcel or esbuild just on that CJS dependency (rather than as a top-level bundler for your whole project), and then cache the ESM result somewhere. If the package is that old and that unmaintained you can just about cache those ESM builds indefinitely. (That's what Vite kind of does under the hood. That's what snowpack used to do.)

I think npm should probably support doing that at install time.

ESM importing CJS works in Node, mostly, now, but it does have quite a bit of runtime overhead and prebaking it would be good. Especially because it is unlikely to ever see a CJS loader in the browser (and that would be awful if it did exist).

if you use a package manager like `pnpm` you can use the `.pnpmfile.cjs`[0] to intercept packages and add `"type": "commonjs"` to the package.json.

This tells node that it needs to load it as a CommonJS package and should work fine with ESM.

There's also creating a require function from `node:module` package[1]

[0]: https://pnpm.io/pnpmfile

[1]: https://nodejs.org/dist/latest-v18.x/docs/api/module.html#mo...

They may be fine.. but nodejs still makes working with them at the cli particularly inconvenient for me. I'm sure there's rational reasons they've made the choices they did, like import being async and which lexical context you're in being significant, but they create an inconvenient mess when I'm trying to test and refactor code around.

Meanwhile, import cycles are easy to design around and to fix with a single interstitial module if you need it, and "live value" exporting has never been an impediment to me. You can export objects, and their properties are "live" enough for me.

Outside of one specific problem in browsers, I'm not sure what ES6 is actually supposed to buy me. I'm still trying to figure out how to turn of ts-ls "File is a CommonJS module; it may be converted to ES6" disagnostic.

I'd be curious to learn more, got any tips? I still export all my Node.js packages in CommonJS so they're usable and I'd love to switch over as well. (I'm already pure ESM in the frontend at least.)

I suggest just adding "type": "module" to your package.json, then you just need to learn the ESM parts of the new "exports" syntax in package.json, ripping the band-aid off, and only shipping ESM today. That limits the low-end version of Node you can support with your package, but that water line is already now below LTS.

> "type": "module"

I've tried using this in the past and it didn't go well, so I suppose support is better these days. I'll give it another shot! Thanks for the tips!

One trick that works better now than at first is that if you need to fallback to CJS for a config file for a CLI tool because it doesn't yet understand the ESM loaders in Node you can just give individual CJS files the .cjs file extension.

I like ESM being the default for .js (which is what "type": "module" mostly does) in a project and then using .cjs files sparingly when necessary (which seems to be mostly just config files for build-time/dev-time tooling with older CLIs today using deprecated loading APIs). That better reflects which is the "present" of JS rather than its past and no need to worry ever about the .mjs bandaid file extension.

Basically what WorldMaker said. We updated our tsconfig and package.json, updated all our imports to include ".js", and fixed/upgraded things as necessary. The only real hiccup was that `require.main === module` no longer works, so we had to write some workaround code to determine whether a script was being run directly.

I'm new to the Node ecosystem but still find it a big pain to get ES Modules working properly with Typescript and `ts-node`, and only a minor pair to get it to work with a Typescript build/run workflow. I have no problems with this feature being experimental but I wish the ecosystem acknowledged it more in documentation.

try giving tsx a try instead of ts-node: http://npmjs.com/tsx

Thanks for this. Just a few minutes ago I pulled up an old project using `ts-node` with ESM and tried to run it on a new machine, had some issues and remembered reading your comment here earlier. I switched to `tsx` and in less than a minute everything was working beautifully. I'll probably migrate all my TS projects to `tsx`.

We're still getting there. Firefox only got the code to allow workers to use modules a couple months ago, and it's currently disabled by default.

I currently have some very exciting code that dumps constructors to text form and sends them to a worker to run...

> dumps constructors to text form and sends them to a worker

Oh do tell more, that sounds devious.

It's mostly a series of `foo=${JSON.stringify(foo)};` and Bar.prototype.constructor.toString(). There's one object that didn't cooperate with either of those, but it's only used in a very limited way, so I did a quick hack to pack its prototype up. At the end is a function that sets up self.onmessage, and a line that executes that function.

Then I do .join('\r\n') on the mess, turn it into a Blob, and pass the URL of the blob into new Worker().

The code running in the worker is only a few classes so it's not worth the hassle of setting up code-copying workarounds or avoiding modules for that portion of the code.

It'll be nice when I can cut that and have nothing more than a few imports and self.onmessage.

I see, a wonderous hack. Sounds like something a build tool/plugin or code transformation step could make it work magically behind the scenes. It's cool and also a bit horrifying (haha) that this is possible dynamically, to stringify a function and send it over to a worker thread to run. After all, code is data..

(2018)

Search for "live bindings" if you don't know what difference those make between require and import. Also explained here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Node should've offered an extended LTS support for a final CJS supported release and just ripped the band-aid off, making ESM first-class for all subsequent releases. As Mike Ehrmentraut once said, "no more half-measures".

I think the pain that the python 2 to python 3 upgrade caused would discourage this.

I also see many people complained about the removal of the long-deprecated egrep and fgrep commands.

The problem is that for many use cases, ESM is not feature complete. There are important features that are under experimental flags or broken in node.js. node.js released broken, backwards incompatible version of ESM with new resolution algorithm and is waiting for everyone to figure out workarounds for theirs shitty implementation.

  [jest ESM support] https://github.com/jestjs/jest/issues/9430
  [typescript extensions] https://github.com/microsoft/TypeScript/issues/16577
  [typescript exports] https://github.com/microsoft/TypeScript/issues/33079
  [node.js issues] https://github.com/nodejs/node/issues?q=is%3Aissue+is%3Aopen+sort%3Acomments-desc+label%3Aesm

Only in the from few months ago, tooling support is close enough to make it usable.

Notably no browser has we workers or service workers of any sort which are ESM capable. Fuck a duck.

Nice pictures

Modules: one of the great programming innovations of the 1960s, first available in JS just fifty years later.

This doesn't really make sense outside of "JS suxx amirite fellas?"

JS had modules before ESM. ES modules try to solve different problems and offer new solutions with browsers fetching modules themselves, something other module systems don't even do.

Whether a language has modules or not is just a design / ecosystem decision, not a technological one. Swift is a modern language (2014) that doesn't even have modules comparable to what Node.js introduced a 14 years ago. Yet it does little to hamper the quality of apps people are building for the Apple ecosystem.

Considering JS didn't exist until the mid 90's, and it's usage was quite limited until the mid oughts, that's not surprising.

I'd say not so much because of the timing but because JS was only meant to be a crude stopgap: the scripting language they'd use for a year or two and then replace. The rest is history, the history of a huge sector of our industry built on a hilariously inadequate foundation.

That's a bit ahistorical. JS was Netscape's scripting language and was supported in both their browser and server. The name change only happened because Java had arrived on the scene and Netscape cut a deal with Sun so JS would not be seen as a direct competitor to Java, the "real" programming language for browser applications. That it was pretty much rushed out the door was more of a product of the times it was born into, namely the browser wars between Netscape and Microsoft.

Microsoft tried to push VBScript as a competitor to JavaScript and ended up copying JavaScript's implementation as JScript. Netscape desperately tried to standardize JavaScript to give it legitimacy (which is how we ended up with ECMAScript, due to the trademark limitations and ECMA being the only standards body that didn't require a lengthy application process).

If you think JavaScript's history is bad, don't look too closely at HTML, a language designed so academics could share information with each other. The Semantic Web pretty much died when search engines became a thing. There's entire offshoots to XHTML that make ES4 look like a success story.

EDIT: It's worth remembering not only what the world looked like that JS was born into but also how the entire Internet evolved since then. Due to JavaScript's unique position, it is heavily invested in backwards compatibility. Except for a number of security-related breaking changes and removals of experimental APIs that never caught on, code written in the early days will still run in modern browsers because it has to. This puts a lot of constraints on its design process and evolution and yet we still see the language undergo massive changes over the past years.

Would it be better if the language stopped evolving?

inadequate for what? js excels far beyond its intended purpose

its my experience that you can say this for most things the non-academic community does. this isn't a criticism of them or you for saying it, but its p constant. the worst offenders in my experience are AAA game devs. ive had so many conversations where they imply they've solved some deep computing problem that was formally solved in 1954 by a set theorist or linguist heh

Reply on news.ycombinator.com