Hacker News Reader

Comment by tosh | Hacker News Reader

> Most notably, the model allows Google to avoid the "diamond dependency" problem (see Figure 8) that occurs when A depends on B and C, both B and C depend on D, but B requires version D.1 and C requires version D.2. In most cases it is now impossible to build A. For the base library D, it can become very difficult to release a new version without causing breakage, since all its callers must be updated at the same time. Updating is difficult when the library callers are hosted in different repositories.

View on HN

Except they don't entirely. As soon as the monorepo interfaces with the rest of the world (like with npm packages, let's say), the problem just gets shifted onto the person trying to import/update a package. Which is further hindered by policy that discourages more than N versions of any given package.

Not that I'm against the policy or how it works. It just isn't "avoided" precisely.

I wonder how Googlers decide what ends up in the monorepo and what is outside of it. Collaborative open source outside (e.g. Android, Fuchsia, Chrome, …)?

It's often the product team's decision. Anybody can spin up a Git repo when they need one. There's freedom there, but you miss out on a lot of tooling and healthy discipline by doing so.

Once you're in the mono repo, you have to play by its rules (well...should anyway). Like in the case of npm packages I mentioned above.

Thanks for the context!

Presumably google doesn’t use npm or other external package managers that download random things from the internet. Why would they?

This seems to me, to be a package manner implementation detail. The packager manager I am most familiar with, npm, won't have an issue with the diamond problem, since node modules can install nested modules, allowing B and C to both depend on unique versions of D.

Effectively rendering B and C incompatible in sharing D objects and structures. Just because you technically can doesn't make it a good thing in software architecture.

There is a reason why most globally linked runtimes (e.g. C++ or Java) do not encourage such mechanisms.

In times of security fixes sometimes there is only one secure version but the whole ecosystem is built around the fact that older, flawed version can still be included. Another drawback is performance, npm install is amazingly slow. In fact package resolution is NP-complete: https://research.swtch.com/version-sat

I generally agree but the problem is even worse :)

Yarn is a bit more sophisticated and has a concept of "resolutions" which would be great for the security use case of forcing a particular version. Even if B and C need different major versions, say D@v1-with-security-fix and D@v2-with-security fix, yarn supports that too, since resolutions can be nested.

https://yarnpkg.com/lang/en/docs/selective-version-resolutio...

What happens when B and C need to pass each other an object from D that's been changed?

It's difficult in one repo too, although for different reasons. To update a dependency you may have to fix every caller in one very large change.

Basically the more users a single dependency has, the harder it gets to update.

Definitely not easy (& might yield to friction / demotivation).

The flip side is that the monorepo (more easily than otherwise) allows the person doing the change to bear the burden of updating the call-sites (& users).

Thanks for pointing out the difficulty this introduces.

paraphrasing Perlis:

Programmers know the value of everything and the cost of nothing.

Reply on news.ycombinator.com