With the addition of workspaces and yarn.lock support to npm 7, are there still reasons to use yarn over npm?
Now I wish yarn could be deprecated and we could go back to a single package manager. There's unfortunately segmentation in different areas around package managers, e.g. electron seems to prefer yarn. And for package maintainers there's extra overhead to document and test installation with both npm and yarn.
We have a monorepo with 180 packages here. Without pnp, it takes 1h+ just to npm install any new third party package in any local package, it’s a joke. With pnp it takes 18s.
So yes, from my point of view NPM is completely inadequate for any serious JS codebase.
I've been downvoted at times for using it to mean exactly that, but I can't help it after more than 40 years of thinking in a language different from English.
But if you start an ambitious company with a JS codebase today, start with yarn v2, you’ll save yourself some pain in the future.
https://guides.sonatype.com/repo3/quick-start-guides/proxyin...
- pnp is made possible in part by mysterious “patches” to certain dependencies that don’t work well with it. Mysterious as in they’re obfuscated, and there isn’t much detail besides commit history. This is blocking if you wanna try out, say, the TypeScript 4.1 beta and the patch isn’t ready yet. But more importantly um... I do not want my dependency manager mysteriously patching stuff with obfuscated code?????
- it applies these patches even if you disable pnp, so same objections to the entire yarn 2 approach (currently)
So I’m back on yarn 1 and apparently gonna need to look at npm 7 at this point.
That said, I think yarn 2's PnP + zero installs (https://yarnpkg.com/features/zero-installs) is lovely with CI. Instead of tacking 40+ seconds to resolve deps every build, vending deps with PnP on is much cheaper than the node_modules equivalent.
FWIW, if I wanted to confirm whether an "unplugged" package had been modified, I'd just download the original tarball from NPM, extract it, and diff the two folders.
The "base64" bit is referenced here [1].
I would assume this specifically relates to the fact that TS does not have native support for Yarn PnP as a filesystem approach. The Yarn team has been keeping an open PR against TS [2] and trying to convince the TS maintainers to merge it, but it hasn't happened yet.
A bit odd, and I can understand why you're concerned, but it also looks like there's a very understandable reason for this.
I would have assumed that this doesn't get applied if you install TypeScript via the Yarn v2 `node_modules` linker, but would have to try it out and actually see.
[0] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...
[1] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...
Note that it gets applied regardless of the linker since it would cause the cache to change when going from a linker to another, and we wanted to make the experience smoother, but that it's a noop for non-PnP environments.
Turning that into a blob is even more discouraging.
Maybe it's just me but a monorepo with 180 packages sounds like a hole you've dug yourself into and you're propping yourself up with yarn.
I certainly don't think that anyone who keeps their packages separate (you could do that even within a monorepo, surely) has a "non-serious" codebase.
I’m not saying the situation is completely perfect (yarn v2 had its rough edges in the beginning for example), but it’s not too bad either. This monorepo is the best organized codebase of this size and diversity I’ve ever seen.
Feel free to explain alternative methods to manage 180 packages with 7 developers while sleeping at night.
Some issues we ran into:
- it can be difficult to reason about the layout of peer dependencies. Often times libraries that rely on Symbols or referential equality break and you need to mess with packageExtensions or add resolutions or unplug. Debugging mostly consists of throwing stuff at a wall and see what sticks
- file watching at large enough projects breaks w/ file descriptor exhaustion errors, forcing you to find and unplug the offending libraries
- there's a number of known incompatible libraries (flow being one of the most prominent) and the core team approach to these at this point follows paretto (20% effort for 80% results, e.g. special casing for typescript), meaning I don't believe there will ever be a 100% compatibility milestone w/ the ecosystem
- it's much more black-boxy in terms of day-to-day debugging (e.g. it's much harder to manually edit files in node_modules to trace some root cause)
- we ran into inscrutable errors deep in packages that interface w/ C++ and basically were only able to fix by pinning to an earlier version of a library that did not depend on said package.
- migration cost is heavily proportional to codebase complexity. My understanding is that Facebook gave up on it completely for the foreseeable future and ours has similarly been a significant time investment (and we're not even targeting strict mode yet)
The pros:
- install times and project switching times are indeed fast, even in our codebase that contains multiple major versions of various packages
- yarn v2 has many interesting features, such as protocols (though it's debatable if you want to commit to lock-in to benefit from those)
I'm not saying it's impossible to debug, just that you end up having to jump through more hoops.
at least this way it's just deleting the temp package folder or running whatever the "replug" command is, instead of having to go figure out all the files you were editing.
You can't accidentally commit your debugging (unplug edits package.json and there's no replug command) and you don't end up with 3 unplugged folders for the same package (that's a whole can of worms on its own). There's also some yarn 2 specific pitfalls regarding __dirname in local packages, symlinking semantics, etc.
Anyways, getting way too into the weeds here, I better stop now lol :)
Mind you, I understand that there are legitimate reasons to approach it this way now (e.g. technical limitations, differences in opinion wrt project governance, cost/benefit on long tail, etc). I'm mostly cautioning the unaware that one shouldn't necessarily expect that every package will work under yarn v2 (though an overwhelmingly large majority does work just fine).
> The package-lock.json file is not only useful for ensuring deterministically reproducible builds. We also lean on it to track and store package metadata, saving considerably on package.json reads and requests to the registry. Since the yarn.lock file is so limited, it doesn’t have the metadata that we need to load on a regular basis.
So I guess there are some performance benefits with npm 7 compared to Yarn 1?
If you just use "yarn" as you'd think of it, you are probably still using Yarn 1, so I guess it's being thought of as a different parallel project
Ships with Node.
OTOH, yarn handles this just fine.
Have you encountered any regularly occurring issues or headaches regarding pnpm?
The immediate issues I ran into were lack of Yarn v2 support for some features critical for internal enterprise usage: no support for the `strictSsl` / `caFile` config options from NPM / Yarn v1, and an inability to read lockfile URLs that were pointing to an internal NexusRepository instance for proxying NPM package installation.
Both issues were resolved very quickly by the Yarn team. I then ran into a problem where the post-install build steps could not run in a locked-down corporate security environment, and that issue was also addressed very quickly, with the Yarn team putting up a PR that tried different process launching approaches and iterating until one worked for me.
Having sorted out those issues, I was able to move on to actually following the steps in the Yarn v2 migration guide [0]. The steps worked basically as advertised. The `@yarnpkg/doctor` tool identified several places where we were relying on imports that hadn't been strictly declared, so I fixed those. Starting up the app caused some thrown errors as other non-declared imports were hit, so I kept iterating on fixing those.
I also used the `@yarnpkg/pnpify --vscode` option to generate some kind of settings file for VS Code, and added the suggested "zip file system" extension to VS Code. That allowed me to right-click a library TS type, "Go to Definition", and show a file that was still packed in a package tarball.
I had to switch off to other tasks and haven't had time to go back and finish trying out the migration. But, parts of our codebase were running correctly, and it looked like I just needed to finish out the process of checking for any remaining non-declared dependencies.
Can't vouch for how this would work out in production or a larger build setup, but things looked promising overall.