Hacker News Reader

Comment by nicoburns | Hacker News Reader

I'd be interested to know what HN's experience with Yarn 2 has been. Yarn 1 was compatible with NPM, so switching was easy, but Yarn 2 seems to need explicit support in a lot of cases.

Had anyone found significant benefits from switching to yarn 2? And conversely, has anyone had any issues with this?

EDIT: It seems that there is also a 3rd option: pnpm. Experiences with pnpm would also be welcomed.

View on HN

We have a monorepo with 200packages. NPM install times were longer than 1hour. Unusable. Yarn v2 took this down to 20 seconds. Yes, around 200x faster. They just are not comparable on that aspect.

The issues we had are with some packages who don’t list their peerDependencies correctly, or which assume the file structure of node_modules will be the one made by npm. But yarn has improved and these obstacles are much easier now.

That is so appealing... I don't have a monorepo with 200 packages but I do `yarn install` in a Dockerfile and if something changed that uncached that line, the whole thing can take 10 minutes or more to download, unpack, install. Would love to try Yarn 2 (3, now, I guess) when I can.

Does Rome use yarn? Anyone using Rome in production?

I haven't actually used Rome, but their installation guide mentions both yarn and npx, so that possibly means Rome is agnostic on them

https://rome.tools/#installation-and-usage

If you are on mac, this is a known docker issue because in non-Linux systems fs access is virtualized (docker runs on an Ubuntu running on a vm). Becomes pretty unusable.

Not Ubuntu, but a very stripped down Linux install.

Thanks for the clarification!

When doing dev you can specify the workspace command and cache any node_modules etc in volumes locally which greatly increases local development speed when changing sepa.

Eg no Docker file in dev unless you need custome system dependencies. Otherwise alpine node in Docker compose and in prod Docker file

We also had a >10x faster experience with Yarn.

I don't understand why npm just can't improve perf. Seems fixable. In my smaller personal projects without as many dependencies, I just use npm and it's easier.

I think part of any (perceived?) performance advantage comes from not being as safe as npm in terms of dependency resolution and lock file preciseness. I seem to get very little out of Yarn, considering its downsides, while npm on the other hand is the standard package manager and simple to use.

Full disclosure, I don't have any experience with yarn aside from personal experiments, so I can't compare yarn with npm fairly. But is it possible that another package manager is even less safe than npm regarding dependency resolution?

Ask your team the question: what's the difference between npm update, npm install, npm update --workspaces, npm install --workspaces, and how the outcome differs between npm versions.

Bonus points for knowing in which scenario package-lock.json is taken into account.

If yarn is even worse, my last hope is gone. Please don't take that post too seriously :)

Right? I'm a web developer and work a lot with PHP.

PHP's de facto package manager is Composer and it's very simple and clear in how it works:

Your composer.json states your dependencies and their version constraints.

Your composer.lock (also a json file) states the actual versions that should be installed, based off your composer.json.

"composer install" installs the exact versions from your composer.lock file, "composer update [package]" updates the lock file based on your constraints.

With npm this doesn't seem to be as straight-forward, sometimes I run "npm install" and the package-lock.json ends up changing, I definitely don't consider npm to be safe.

This is why you should use yarn v2 or v3. Dependencies clusterfck are not a thing with yarn. We went from having a lot of weird bugs like that every month to having zero, for 2 years thanks to yarn. The counterpart is that yarn sometimes needs your input, when package maintainers don’t declare their does or peerDeps correctly.

Is that due to yarn v2 including a zipped version of all the dependencies that you can check into the repo?

Because that is what I plan to do. Such a pain when some random dependency 50 packages deep is broken or even pulled from npm and so we can't even finish a deployment build until fixed. Especially for older projects.

It’s is first and foremost due to the “mathematically correct” (or at least “more correct”) resolution algorithms used in yarn. And it is further improved by the local caching of packages indeed. Note that you can even be in “zero install” mode with yarn, where you checkout in git this dependency cache, so you never have to “yarn install”, got checkout is enough to get everything. If npmjs.com falls down you wouldn’t be in any trouble.

Composer is not that much better though. It has (had?) the brain dead convention, that you may only refer to branches by prepending "dev" or "dev-", which itself can be a branch name and an organizational convention and thus the convention prescribed by composer gets in the way unnecessarily with what an organization may already have as convention.

So I am not sure it is better than npm, however, at least one does not have that split in package managers, which will additionally create friction in or between teams and projects.

npm install updates the lock file when you have changed a dependency in package.json, which is reasonable behavior.

It’s not if you want your install command to be deterministic.

I never want the install command to change anything. I want it to install only.

> I run "npm install" and the package-lock.json ends up changing

This is intended behaviour, but seems totally counterintuitive to me. I’m with you, I’m used to Composer and NPM just seems inscrutable at times.

For reference, the approximate equivalent to `composer install` is `npm ci`. This will install the exact versions from package.lock without changing it, however it will also blow away your node_modules directory and install from scratch each time.

My experience is yarn is safer in terms of locking as well, I had a lot of subpackage versions suddenly changing with npm causing CI tests to fail whereas with yarn it's very rare.

Yarn is absolutely safer than npm in terms of determinism and lock file preciseness. I think it was actually one of the main goals of yarn v2 to be fully reproductible and deterministic.

After performance, the other main reason we moved from npm to yarn was exactly that it completely removed an entire class of bugs we had that were caused by incompatible packages versions installed in various places of the repo.

In exchange of that, yarn will warn if it encounters packages that don’t declare their reps and peerDeps correctly. And there are loads of those. if you want to be safe and remove these warnings, you will have to either ask the package authors to declare their depa correctly, or fix them yourself manually in your yarn YML file.

You've got it backwards. npm has historically been much less deterministic than yarn, and recently decided auto-installing peerDependencies was a good idea...

Because of cocky maintainers

NPM has gotten faster over the last few years. When Yarn first came out it was dramatically faster than NPM. NPM 5 then caught up with Yarn 1 on performance, and NPM 7 is often faster than Yarn 1. NPM has not caught up with Yarn 2, but Yarn 2 had to break a lot more things than NPM is willing to break.

My experience with Yarn 2 was switching to NPM 7.

NPM 7 is fantastic, and feels like a worthy successor to Yarn 1. Yarn >=2 is probably fine for what it is, but I wasn't prepared to invest resources into rearchitecting how we handle dependencies for nebulous (or possibly negative) benefit. Meanwhile, the old release of Yarn 1.x that I'd pinned in order to work around a blocker regression (v1.21.1) was starting to show its age, and eventually became unusable after a third-party package update triggered another hidden blocker bug in that release (which, IIRC, the team had elected not to fix because only 1.x was affected). At that point, it was untenable for us to continue relying on a broken, unmaintained tool for such a critical function.

Luckily, the stable release of NPM 7 arrived just in time for us to switch over with minimal hassle. Performance and reliability have been notably better in my experience (keeping in mind that my baseline for comparison is a release of Yarn from two years ago).

I'm grateful to the Yarn project for pushing NPM in the right direction, and wish it and its remaining users all the success in the world, but I no longer see an urgent need in the ecosystem for Yarn to exist and would no longer recommend it by default for any greenfield project.

My team upgraded our frontend monorepo from npm+lerna to Yarn 2 with PnP. We saw vastly improved installation time, most importantly on incremental installations. The startup times for jest, webpack, and TypeScript also improved. PnP's dependency strictness eliminated cases where updating a package in one workspace would break a different workspace.

It was a fairly difficult migration, in particular because Yarn 2 PnP does not allow you to import a module that is not specified in package.json. Lots of libraries play fast and loose with dependencies, and so our .yarnrc.yml file which adds those missing dependencies is now 172 lines. PnP requires configuration for vscode and a custom build of TypeScript, which isn't ideal, but those have worked well.

Yarn 2 is fantastic.

I run about ~30 JS repos, ~10 of them monorepos. Since I migrated everything to yarn 2 its been a breeze, my other contributors have been able to completely forget about it (as it just works) and I've been able to assert that everything that works today will work in six months and two years.

That is because I have not gone against the default of 0-config repos, that is saving the .yarn/cache folder (equiv of node_modules) to git (via git-lfs). The lockfiles work. Dependencies can have conflicting dependent packages. I've had a small number of packages have issues with the zip format, which is solved by setting unplugged (very well documented).

A user clones the repo and they are done. Yarn itself is saved into the repo, so the only external dependency is nodejs.

Things I haven't thought about or done since switching to yarn 2:

* I don't worry that the server is running something different to the dev machine

* I don't get breakage because I switched branches

* I don't have to teach other people how to look after their packages (because it 'just works')

* I don't have to chase people to update when a lib we use has a vuln

* I don't spend time installing

* I don't care about package compatibility

Things I have had to worry about:

* build systems. When you use webpack, esbuild, etc. then they need a little help resolving packages

* in the beginning it was hard to step into dependencies. That is all fixed now.

* Sometimes a package is broken because they don't specify peerDependencies properly (which is a security & reliability issue), so you have to add to a config file (surprising first time, easy from then on).

Yarn is a big win.

Does this strategy work with node modules that use native extensions, such as node-gyp?

Yes completely. No problem with those.

Yarn 2 has killer features

- environment-independent lockfiles

  - yes, you can have the same lockfile and run it behind different proxies with the same result now! very strong for corporate proxy situations with external dev teams

- pnp

  - superbly fast package install, gets faster as you use it more

Yarn 2 also has an upfront cost

- all new config (yml)

  - also renamed pretty much every property

  - docs are decent and mitigate some of this

- doesn't come for free with node.js

- conflicts with npm with regards to commands

- interacts awkwardly with npm config, leading to some situations where yarn installs fail due to npm config being partially and implicitly merged

Overall it's pretty good. Not really sure it's worth it unless install performance or multiple-proxy situations are present at your workplace though.

I manage several monorepos with PNPM's workspaces. One of which is an open source repo for a popular bundler which accumulates 100s of millions of weekly downloads. Another is internal and private that contains an entire organization's codebase. PNPM is an absolute joy to work with. If you haven't tried it, make a point to.

FWIW I tried pnpm 8months ago and had problems with a lot of tools/packages. Next (or some next related project) did not work well with it, and many others. Went back to yarn v2.

I'm aware of symlink issues with React Native, but none from Vercel, Next, etc. The issue was more likely bundler related.

You’re right, it was probably with our react native package, and maybe also with our electron package. And also with Strapi. And maybe others, I will never know.

That's one thing that is unfortunate about pnpm - the methodology is superior in many ways, but there are a lot of tooling packages that don't consider symlinks for dependency resolution because npm and yarn are so prevalent.

Echoing all current replies, plus another thing not mentioned:

yarn.lock is _at least_ 15x smaller than an NPM lockfile. You want this committed into your repo, and I'd rather have ~200kb than 5mb.

So that's:

- Speed

- Artifact size

- Caching

- Proxy support

- Determinism (which has improved with NPM)

I had issues with Yarn 1, when it came to concurrently managing its own cache, when I had packages, which have dependencies to private git repos, which could be referenced using multiple ways of writing their URLs. In such cases one had to manually make sure, that the order of installed packages is exactly right, otherwise the cache gets into problems and errors would appear. Afaik that issue never got resolved, except perhaps by accident in a newer version of Yarn, but no explicit treatment of admission of the issue.

To me it was outrageous, that a tool used by so many could have such flaws. I stayed with npm, which also has the advantage of storing a more precise lock file than Yarn and I never had the same kind of issues as I had with Yarn. It seems more standard and safer than Yarn ever did, so I never bothered with upgrading Yarn.

I often need to work with a project, which has a big monorepo and chose to bundle its version of Yarn with that. Now it seems that there will forever be a Yarn version inside that project.

I have not noticed that much of a speed difference between npm and Yarn once things are cached and I value safety and correctness more than speed anyway, so that has not been a reason for me to try and make the switch either.

I've found two major benefits with yarn. First is package install time, like others have said it's an order of magnitude faster for projects I've worked on.

Second is a little more niche in running a Linux image on Windows host via Vagrant/Virtualbox. Out of the box symlinks won't work so bin files often fail. I would also run into file locks when running npm install on larger projects often, to the point where I was having to run it 5-10x before everything would finish installing. You can instruct Vagrant to offer some support (setextradata VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root 1), or set an environment variable to key on and update any scripts to use --no-bin-links, but it's flaky in my experience. I never found a solution to the file locks with npm.

I installed yarn on a whim on that setup and have yet to have any of those issues with it.

Even for a smaller project, on yarn v1, our installs was like 3 minutes. With yarn v2 it is basically instant - saves so much time!

I can also add that the monorepo setup really makes it easy to manage project with multiple packages and interdependencies. You can have a look at our config for this here: https://github.com/lowdefy/lowdefy

I tried it once but installation was incredibly slow. I had more luck with PNPM.

For standalone projects there’s probably no benefit to be had.

It's basically not supported in Ember yet, so.... Not at all. My fingers are crossed for pnpm support soon so I can be done with the sadness pile that has been yarn as of late (e.g. linking just flat out doesn't work for us with Ember addons).

My experience is that Yarn 2 works great as long as you stick with the `node_modules` linker. It's the "Plug 'n Play" functionality that is great in theory but still rough in practice.

Reply on news.ycombinator.com