Node.js 15.0
nodejs.medium.com
nodejs.medium.com
process.on('unhandledRejection', (err) => {
console.error(err.stack);
process.exit(1);
});
In Emscripten we've had to emit a handler like that by default, so that test suites don't silently ignore errors. Being able to not emit it will avoid some code size and annoyances users have had.
we instead use
process.on('unhandledRejection', (reason) => {
throw reason;
});
https://github.com/nodejs/node/blob/master/doc/changelogs/CH...
It turns out that a content blocker in Safari was causing this. I have no idea why, perhaps because of some class name or ID in the HTML on Medium?
If you just use "yarn" as you'd think of it, you are probably still using Yarn 1, so I guess it's being thought of as a different parallel project
> The package-lock.json file is not only useful for ensuring deterministically reproducible builds. We also lean on it to track and store package metadata, saving considerably on package.json reads and requests to the registry. Since the yarn.lock file is so limited, it doesn’t have the metadata that we need to load on a regular basis.
So I guess there are some performance benefits with npm 7 compared to Yarn 1?
Now I wish yarn could be deprecated and we could go back to a single package manager. There's unfortunately segmentation in different areas around package managers, e.g. electron seems to prefer yarn. And for package maintainers there's extra overhead to document and test installation with both npm and yarn.
Have you encountered any regularly occurring issues or headaches regarding pnpm?
The immediate issues I ran into were lack of Yarn v2 support for some features critical for internal enterprise usage: no support for the `strictSsl` / `caFile` config options from NPM / Yarn v1, and an inability to read lockfile URLs that were pointing to an internal NexusRepository instance for proxying NPM package installation.
Both issues were resolved very quickly by the Yarn team. I then ran into a problem where the post-install build steps could not run in a locked-down corporate security environment, and that issue was also addressed very quickly, with the Yarn team putting up a PR that tried different process launching approaches and iterating until one worked for me.
Having sorted out those issues, I was able to move on to actually following the steps in the Yarn v2 migration guide [0]. The steps worked basically as advertised. The `@yarnpkg/doctor` tool identified several places where we were relying on imports that hadn't been strictly declared, so I fixed those. Starting up the app caused some thrown errors as other non-declared imports were hit, so I kept iterating on fixing those.
I also used the `@yarnpkg/pnpify --vscode` option to generate some kind of settings file for VS Code, and added the suggested "zip file system" extension to VS Code. That allowed me to right-click a library TS type, "Go to Definition", and show a file that was still packed in a package tarball.
I had to switch off to other tasks and haven't had time to go back and finish trying out the migration. But, parts of our codebase were running correctly, and it looked like I just needed to finish out the process of checking for any remaining non-declared dependencies.
Can't vouch for how this would work out in production or a larger build setup, but things looked promising overall.
We have a monorepo with 180 packages here. Without pnp, it takes 1h+ just to npm install any new third party package in any local package, it’s a joke. With pnp it takes 18s.
So yes, from my point of view NPM is completely inadequate for any serious JS codebase.
I've been downvoted at times for using it to mean exactly that, but I can't help it after more than 40 years of thinking in a language different from English.
But if you start an ambitious company with a JS codebase today, start with yarn v2, you’ll save yourself some pain in the future.
https://guides.sonatype.com/repo3/quick-start-guides/proxyin...
- pnp is made possible in part by mysterious “patches” to certain dependencies that don’t work well with it. Mysterious as in they’re obfuscated, and there isn’t much detail besides commit history. This is blocking if you wanna try out, say, the TypeScript 4.1 beta and the patch isn’t ready yet. But more importantly um... I do not want my dependency manager mysteriously patching stuff with obfuscated code?????
- it applies these patches even if you disable pnp, so same objections to the entire yarn 2 approach (currently)
So I’m back on yarn 1 and apparently gonna need to look at npm 7 at this point.
That said, I think yarn 2's PnP + zero installs (https://yarnpkg.com/features/zero-installs) is lovely with CI. Instead of tacking 40+ seconds to resolve deps every build, vending deps with PnP on is much cheaper than the node_modules equivalent.
FWIW, if I wanted to confirm whether an "unplugged" package had been modified, I'd just download the original tarball from NPM, extract it, and diff the two folders.
The "base64" bit is referenced here [1].
I would assume this specifically relates to the fact that TS does not have native support for Yarn PnP as a filesystem approach. The Yarn team has been keeping an open PR against TS [2] and trying to convince the TS maintainers to merge it, but it hasn't happened yet.
A bit odd, and I can understand why you're concerned, but it also looks like there's a very understandable reason for this.
I would have assumed that this doesn't get applied if you install TypeScript via the Yarn v2 `node_modules` linker, but would have to try it out and actually see.
[0] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...
[1] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...
Note that it gets applied regardless of the linker since it would cause the cache to change when going from a linker to another, and we wanted to make the experience smoother, but that it's a noop for non-PnP environments.
Turning that into a blob is even more discouraging.
Maybe it's just me but a monorepo with 180 packages sounds like a hole you've dug yourself into and you're propping yourself up with yarn.
I certainly don't think that anyone who keeps their packages separate (you could do that even within a monorepo, surely) has a "non-serious" codebase.
I’m not saying the situation is completely perfect (yarn v2 had its rough edges in the beginning for example), but it’s not too bad either. This monorepo is the best organized codebase of this size and diversity I’ve ever seen.
Feel free to explain alternative methods to manage 180 packages with 7 developers while sleeping at night.
Some issues we ran into:
- it can be difficult to reason about the layout of peer dependencies. Often times libraries that rely on Symbols or referential equality break and you need to mess with packageExtensions or add resolutions or unplug. Debugging mostly consists of throwing stuff at a wall and see what sticks
- file watching at large enough projects breaks w/ file descriptor exhaustion errors, forcing you to find and unplug the offending libraries
- there's a number of known incompatible libraries (flow being one of the most prominent) and the core team approach to these at this point follows paretto (20% effort for 80% results, e.g. special casing for typescript), meaning I don't believe there will ever be a 100% compatibility milestone w/ the ecosystem
- it's much more black-boxy in terms of day-to-day debugging (e.g. it's much harder to manually edit files in node_modules to trace some root cause)
- we ran into inscrutable errors deep in packages that interface w/ C++ and basically were only able to fix by pinning to an earlier version of a library that did not depend on said package.
- migration cost is heavily proportional to codebase complexity. My understanding is that Facebook gave up on it completely for the foreseeable future and ours has similarly been a significant time investment (and we're not even targeting strict mode yet)
The pros:
- install times and project switching times are indeed fast, even in our codebase that contains multiple major versions of various packages
- yarn v2 has many interesting features, such as protocols (though it's debatable if you want to commit to lock-in to benefit from those)
I'm not saying it's impossible to debug, just that you end up having to jump through more hoops.
at least this way it's just deleting the temp package folder or running whatever the "replug" command is, instead of having to go figure out all the files you were editing.
You can't accidentally commit your debugging (unplug edits package.json and there's no replug command) and you don't end up with 3 unplugged folders for the same package (that's a whole can of worms on its own). There's also some yarn 2 specific pitfalls regarding __dirname in local packages, symlinking semantics, etc.
Anyways, getting way too into the weeds here, I better stop now lol :)
Mind you, I understand that there are legitimate reasons to approach it this way now (e.g. technical limitations, differences in opinion wrt project governance, cost/benefit on long tail, etc). I'm mostly cautioning the unaware that one shouldn't necessarily expect that every package will work under yarn v2 (though an overwhelmingly large majority does work just fine).
OTOH, yarn handles this just fine.
Ships with Node.
Ryan's original criticisms regarding Node were totally valid, but most of them weren't really easily addressable without significant breakage or a long migration strategy, which potentially could've caused a _lot_ of issues and unclarity for many years.
The top-level comment seems to suggest that Ryan was an integral part of node development before deno and that "getting him back" was relevant. Which isn't what happened. There are people who worked on node and moved to deno. But Ryan isn't really one of them.
1. Not sticking with Promises: This is changing, slowly. You can `import {readFile} from 'fs/promises'` in Node and it works as you'd expect, including top-level await. (Backwards compatibility means the callback API can never go away.)
2. Security (your linter shouldn't have complete access to your computer and network): Deno hasn't done a great job with this, either. You can restrict the access that a Deno process has, but you can't restrict the access for individual modules. If any module in your server needs to access something, then every module in your server can access it.
I predict that module-level authorizations will be solved some day by browser vendors, and that Node and Deno will adopt the thing. Deno will probably have to throw out their thing what that happens.
3. Build system (GYP). This has no effect on userland Node developers. You build node with make. Another build system could be adopted, but I think nobody's bothered. Deno has a protobuf FFI to communicate with V8. You can do that with Node if you want. Shrug.
4. require("package") relies on package.json. Deno uses import maps. Node will probably honor import maps someday, too.
5. node_modules: He said it "complicates the module resolution algorithm." Meh. He also points out that node_modules is too large, but that's a Node cultural problem. Deno's community is still small, but it will have that problem, too, except it will have a large shared "cache" instead of a large local node_modules folder.
6. require("module") without the extension ".js": Deno does this, too, using import maps. It's fine.
7. index.js: Again, it "complicated the module loading system." Meh?
I can't believe I missed that. I've still been writing promise wrappers like a fool.
Perform operation and save value.
I'd argue every operation should have already had it's respective assignment operator to be more consistent.
Life shows that usually one can't catch lightning in a bottle twice.
Logical OR assignment (||=) [1] Logical AND assignment (&&=) [2] Logical nullish assignment (??=) [3]
An example for OR assignment:
let a = '';
a ||= 'hello';
console.log(a); // prints 'hello'
a ||= 'not this';
console.log(a); // prints 'hello'
or have people made nvm run when their system or terminal window starts so that they don't have to remember to do it
UPDATE_NVM='[ -e .nvmrc ] && [ v$(<.nvmrc) != $(node --version) ] && nvm use'
PROMPT_COMMAND="$UPDATE_NVM; ...other prompt commands..."Many of these version managers always simply can't be involved outside interactive shells because they require unit script to be loaded first.
In this state, shouldn't it be Node 0.15?
And, frustratingly, the line where Node ends and frameworks begin are too blurry. This is made worse by the fact that thousands of bloggers have a node server tutorial, essentially drowning out the good ones.
I also don't see how it could be possible to find the line between Node itself and frameworks ambiguous - unless the framework itself is invoking your JS files, in which case I would recommend moving to something lighter (i.e. something more designed for composition - where you call it, rather than it calling you)
You need to know the args, and it just seems "hacky". Like it's good to write something small when you know what functions you use etc... but just not stable for big stuff.
Also, it seems like your comment could be generalized to include all dynamic language runtimes, not just Nodejs.
You can set up most IDEs to get excellent auto completion - VS Code does a good job of that kind of thing.
IDEs do spoil people.
Makes scripting languages really hard for me to use as a consequence.
If anything, TypeScript sometimes feels like a nice middle-ground between C# and JavaScript (and Java?), and though it's not perfect, I do feel that it's pleasurable once you get the hang of it and the quirks of the ecosystem.
The most challenging thing for me in Node right now it upgrading an application with my HTTP services to use HTTP 2 with binary streams.
Isn't that solved with a simple google search? Async/await makes the function return promises, so if you use an async callback in Array#map you end up with an array of promises, which you can use with Promise#all. Seems rather simple to explain and grok.
I'm a scala person, so I naturally tend to like promise style (and async/await is a nice sugar) but my experience interviewing a lot is that people that were not exposed a lot to callback style don't understand what asynchronous means, which is a good enough reason IMO to keep callbacks :) the fact that the std lib is also full of them (we are using node 12, I don't know what is the current status, but for now most of the std lib is callback only for us) does not make me want to do the switch at all.
Maybe during the interview process we should make sure the candidates fully understand the callback style and Javascript's async model.
We also have a big project. we started during callback days, skipped promises (I never thought promises alone were an improvement over callbacks and async.auto). But as soon as async/await was in, we allowed them in the codebase.
So now the codebase is a mess, with some functions being callback style and some others promises, stiched together we promisify functions. Im hoping in a 2-3 years time frame all our callback style functions will be eventually phased out during small refactors and rewrites.
This migration plan is the only method i found that enables projects to move from A -> B without spending massive times rewriting the whole project and yet keeping up to date with technologies so they wouldn't need a rewrite every 10 years.
I used to ask candidates to implement async.map and I used to be baffled that the majority of JS devs are unable to do it. There are those who would say "I'm not used to callbacks, I use promises". I would then allow them to use promises (no Promise.all allowed, that's what we are trying to implement ^^). I have never seen one of those "promise only candidate" manage to do the exercise.
If you want to push it to the next level, ask them to implement async.mapLimit. You wouldn't believe how few people that are looking time professional JS devs, are actually able to do it properly (I actually used to ask that one, before realizing that it was too much to ask... )
All you would have had to do is make it so when you use the await keyword, you don't pass the callback, the runtime passes it for you, pauses the function and resumes it when the callback is called (returning values and throwing errors as you would expect). I implemented this here: https://github.com/bessiambre/casync but it would be even better with language level support.
Whenever I encounter JS beginners and they ask me about simple looping over asynchronous operations sequentially, I mention async/await, the discussion veers towards promises and then I have to mention the state machine and the caching layer for errors and return values, pending, fulfilled, rejected, settled states, reject/resolve callbacks and how it all fits together with 'then' chaining and 'dynamic replacement' by which point they just want to go back to Python or whatever.
With a callback based async/await, I could just say: Here, with this keyword, the function will pause until the callback is called. Just put your function taking a callback in a normal loop and await it. It would be cleaner, more functional, more stateless and faster.
The features of promises aren't even used with async/await. The point of promises, the caching layer and the state machine, is to be able to add the continuation later but with async/await, it is always just the next line in the function so it doesn't need to be added later. It's unnecessary performance penalty, complexity and statefulness on every asynchronous function call.
I did try to propose this as a language feature here: https://es.discourse.group/t/callback-based-simplified-async...
It didn't get much traction an I was too busy to push it further.
I still can't believe the nodejs universe doesn't have back pressure (eg caolan/async) baked into the stack. Those were some pretty difficult conversations. My teammates had no idea what I was talking about or why my fixes worked.
Absolute paths for importing modules (requires) still makes me laugh.
But criticizing nodejs, js, etc is rather pointless. Like PHP, it's fractally wrong.