Comment by lhorie | Hacker News Reader

Comment by lhorie | Hacker News Reader

IMHO, if you're targeting the Javascript ecosystem, this area is already fairly crowded, with Turborepo, Nx and various open source tools providing various degrees of functionality (Bazel, Pants, Lerna, etc) already competing in the space.

I'm a tech lead for the web monorepo at Uber. We talked to the Turborepo guy a few years ago, and he admitted that he wasn't sure if it could handle our scale in terms of all the bells and whistles that a repo of our size leverages - and his is one of the more feature packed commercial offerings in this space.

As a random example: we see thousands of commits a week, so invalidating the whole build graph when the lockfile changes is a complete non-starter, yet most turn-key solutions target small teams and are not equipped to cope with this problem. Phantom dependencies[0] are a problem with disjointed build graphs. Etc.

As someone who's evaluated a ton of these systems, I would love to see where your experience in this space is coming from. A new kid in the block will have a lot to prove.

[0] https://rushjs.io/pages/advanced/phantom_deps/

We agree. We weren't happy with all of the current solutions, at least in the JavaScript space.

In regards to build graph invalidation, we attempt to hash pieces at the granular level. This includes per file content hashing, and for dependencies (those in `package.json`), we parse the lockfile and extract the resolved version/integrity hashes. We can probably improve this further, but this has been working great so far.

As for phantom dependencies, this isn't something moon solves directly. We encourage users to use pnpm or yarn 3, where these phantom problems are less likely. We don't believe moon should solve this, and instead, the package managers should.

If you have any other questions or concerns, would love to hear them.

I guess I'll echo some of the other comments re: extensibility. For us, Bazel's Starlark language is useful because it allows us to integrate w/ non-JS things like protobuf schema definition files. I suspect most companies are going to have hybrid stacks instead of Node.js-only ones. FWIW, Bazel vs Buck have some history wrt adoption wars, and Bazel largely pulled ahead due to a better ability to flourish an ecosystem (hell, the entire JS ecosystem around it is done via the extensibility system). It very much reminds me of how Typescript won against Flow.

> We don't believe moon should solve this, and instead, the package managers should.

My IDE solves this in that I see red underlines, but I also want my build to fail as well. Not everyone uses an IDE that would show this error.

Can you summarize Moon's sandboxing approach? I understand for phantom deps you want to delegate to pnp/pnpm. But how do you handle sources not declared explicitly as inputs to a build graph node?

If I have package A only list its own sources as inputs, how do you prevent the Node runtime from happily doing require('../B')? If you don't, how do you prevent differences in the state of B from poisoning the remote cache?

Turborepo author here…

We do not invalidate the whole graph anymore for a lockfile change. We now have a sophisticated parser that can calculate if a change within the lockfile should actually alter the hash of a given target. In addition to higher cache hit rates, this is what powers our `turbo prune` command which allows teams to create slices of their monorepo for a target and its dependencies…useful for those building in Docker.

Prune docs: https://turbo.build/repo/docs/reference/command-line-referen...

Turborepo is much more scalable now than when we spoke pre-Vercel acquisition. It now powers the core web codebases at Netflix, Snap, Disney Streaming, Hearst, Plex and thousands of other high-performance teams. You can see a full list of users here: https://turbo.build/showcase

Would be happy to reconnect about Uber’s web monorepo sometime.

Can Prune be used to build a bundle (as in a zip) for, say AWS Lambda, which includes only the dependencies (and not dev dependencies)? I've played around with pnpm's deploy but it felt a bit lackluster. Especially talking about situation where one has a backend package and some shared package. The bundle should contain all dependencies (but not dev dependencies) of the backend package and shared package and of course the built shared package should also be include in the bundle's node_modules.

Yes! Prune then zip the output folder.

Kind of. Poster asked if you can prune only deps and exclude dev deps. That's currently unsupported: https://github.com/vercel/turbo/issues/1100

That's correct. However, I've tried to use prune now, not sure if I am using it correctly, but that's why I do:

I build my packages regularly (not pruned), then I prune with scope "backend". Apparently the pruned directory contains node_modules with empty packages, not sure what reason is for that; I just ignored it. In the resulting directory I then run `pnpm install --prod`. Only the regular dependencies will be installed. I think this is enough for my usecase. I am not sure if prune is supposed to be used for this approach though.

You can do that with esbuild. Just bundle the handler entry into a single file and esbuild will tree shake the cruft out. That's the approach taken by AWS CDK as well.

Yeah, that always works, of course, even though you might want to externalize certain dependencies still. In this case I don't want to bundle the code (although I might end up doing that, anyway)

You can give esbuild a list of dependencies that you want to be external and it won’t bundle them, I’ve done it with the AWS SDK and it works as expected.

Nx is hot garbage imo. Buggy, overly verbose, inflexible, poorly documented, and I'm dubious about their peer review process.

If I don't need build caching I'm not using any tool but PNPM and its workspace toolset - that's literally all most people need for a monorepo. I've looked into Turborepo, and its simplicity versus Nx is a strength. However, it's not the taskrunner that I want.

I now work in a monorepo where build caching is required, so I'm excited about moon and keeping a watchful eye on the project's progress. From my evaluation so far, it fixes all of my gripes about Nx and I'm keen on it not trying to do too much, while allowing me to make it as flexible and extensible as I need. Extending configs is chefs kiss

If you end up using moon, and there's anything abrasive/convoluted, let us know! We're always looking for ways to streamline.

Nx is a pita (imo). I have had amazing success with rush and pnpm.

Should give moon a try if you ever have free time :)

Can confirm. Steer clear of Nx.

That is really interesting.

NX has been on my "to learn" list for a long time and high enough that I have given it a few stabs with personal project but something have failed every single time. I chalked it down to my setup or my inexperience but maybe it just isn't as good as I thought.

> However, it's not the taskrunner that I want.

Could you elaborate a bit on what you mean here? Is this saying Turborepo's task runner is flawed in some way?

Reply on news.ycombinator.com