Require(esm) in Node.js | Hacker News Reader

Require(esm) in Node.js

joyeecheung.github.io

34 pointstilt25 comments

Appreciate this change. The ESM stuff is nightmarishly esoteric. Most recently I ran into ERR_REQUIRE_ESM while trying to write Playwright E2E tests that import from other packages in our monorepo.

I really think the way ESM was handled hurt Node.js a lot. Couple this with the introduction of TypeScript and complexity in the Node.js ecosystem skyrocketed. I've lost so many hours on this.

Thankfully libraries like esbuild exist (and by extension `tsx` which uses esbuild under the hood). https://github.com/privatenumber/tsx

> Couple this with the introduction of TypeScript and complexity in the Node.js ecosystem skyrocketed

I remember the days of AMD, UMD and so on and so on. The complexity just got reduced during CommonJS days and then came back with a vengeance.

> After dwelling on this a bit, I think the reason why the synchronicity of ESM didn’t lead to a synchronous require(esm) in Node.js sooner was more cultural than technical. There seemed to be a silo problem between those who worked on the ESM implementation in Node.js/communication with the standard bodies and those who didn’t.

Must be something like that.

This never made a lick of sense.

(At the very least for synchronous modules.)

It's all just so extra crazy because https://github.com/standard-things/esm has just worked for so so long. The intense module discussions kept happening, about why it wouldn't work or how it would blow up, but standard-things/esm was already out & working great.

Awesome that this is in Node.

I created require-esm-in-cjs[1] two years ago when I needed to require ESM modules in a commonJS (regular legacy) page.

It is only a few bytes long and is simple - it effectively causes ESM modules to load synchronously by checking every 100ms if the JS has loaded before allowing the page to continue.

I have been meaning to improve it, such as adding a timeout and whatnot but never got around to it (it is good enough for my needs) - but always thought it was odd that something so (relatively) simple to hack around and so low in the stack can not be supported natively.

It gets a fair amount of downloads even though it doesn't get much visibility so I guess I am not the only one who felt the need. But native is MUCH better!

[1]: https://www.npmjs.com/package/require-esm-in-cjs

I read your readme, clever

By the way, we've been saying this for years. And anyone who pointed out how ridiculous this was got ostracized by Node devs. It's just baffling how this was handled.

I really wish ECMA just ran with the Node.js require instead of unnecessarily creating a new standard. Maybe one day we'll see interchangeable use of require/import.

One has to wonder about the total number of hours wasted debugging issues that trace their origin to ECMA's decision.

Not possible. require() is broken by design by being synchronous.

It’s not though. You can return an awaitable object.

No, async module loading or statically analyzable module loading is necessary for web.

AMD wasn't invented on a whim

> Recently I landed experimental support for require()-ing synchronous ES modules in Node.js, a feature that has been long overdue.

Oh come on, overdue for what purpose? Digging Node out of another Node created hole?

Did you read the article? He explains the motivation for the change pretty clearly.

The motivation seems to be "many people have transpilers configured wrong" and this seems an extreme solution to fixing people's transpiler configurations. Especially because it will still fail for modules using top-level await (anywhere in their dependency graph), so it is at best a band-aid. Beyond that it does seem to be missing real motivating use cases. What real reason in 2024 is there to use CommonJS at the top level in NPM? How hard is it to rewrite `require()` statements to `await import()` ones, especially when the fix is a transpiler configuration change away?

I don't know how much experience you have, but in the JS world there are band-aids everywhere. They "fix" a lot of real-world problems, help push things to production, and are more important than you think.

Just look at the typing gymnastics in TypeScript.

The solution to band aids existing isn’t to introduce more, especially not more that are useless.

1. I don't agree this is useless. You may very likely never need to use it, but not for others. 2. Until every single JavaScript file in the world is in the ESM format, anything that happens before that will be a band aid of some sort.

> I don't know how much experience you have

Try better to avoid ad hominem attacks, please.

> but in the JS world there are band-aids everywhere

I apologize, "band-aid" was the wrong word to use for this specific case, because that unintentionally disparages "band-aids". You are right there are "band-aids" that fix real-world problems, and help heal accidental wounds. This experiment does neither and seems to me worse than a "band-aid". It's a fragile broken and fake bridge that looks like it can get you to your destination but will collapse right under you if you try to use it (rather than switching to what already works today).

Sorry to disappoint you, half of the things in the JS world are half baked, broken, and are fake bridges. That's why I need to mention experience -- it is hard to explain all of this within a few sentences if you don't already have the context of what I am trying to say.

> many people have transpilers configured wrong

It's so that CommonJS modules can use ESM modules.

---

Like suppose examplelib uses otherlib, both in CommonJS.

The examplelib wants to upgrade otherlib, but the new version uses ESM. Now, examplelib is stuck either on the old version, or has to undertake a rewrite (and propagate this situation downstream). Or adopt a transpiler and force downstream users to do the same.

Some would say "just rewrite in ESM" and that is a nice idea, but there are lots of tools with caveats, TypeScript, Jest, anything that depends on require-in-the-middle, etc.

It's hard to boil the ocean. (Just look at Python 3.)

CommonJS modules (and anybody else that prefers or needs "require-in-the-middle") can already import ESM. They need to use `await import()` or `import().then()` if they prefer callback waterfalls to async/await syntax. That works everywhere in Node today. Sure, it's a pain when upstream switches to ESM that you have to use more Promises, but that the price you pay and it is certainly a lot less work than a full rewrite even for the most brownfield CommonJS library. You don't have to rewrite everything to ESM all at once. (You should want to, but don't have to.)

Exchanging one function coloring problem for another isn't what I would call "less work."

It's quite difficult to write require() to `await import()` due to the introducing of `await`... Now everything needs to become async

If your upstream has switched to ESM and expects to do async things, then yes, everywhere you depend on that upstream you need to consider those imports async. Yes, it's contagious, but your upstream already caught that contagion. Trying to keep smashing it into a synchronous box seems like a recipe for unexpected bugs. Especially in the case of this experiment where the difference between "works" and "gives the same error it already gives today" is a one keyword change anywhere in the upstream file. That sounds brittle to me and less useful than "everything needs to become async".

Just not upgrading and use patch-package to apply security fixes work so far.

Reply on news.ycombinator.com