Monorepos are changing how teams build software | Hacker News Reader

Monorepos are changing how teams build software

vercel.com

8 pointsgk119 comments

Having worked in companies with large monorepos and those without, I would never advise implementing one.

The complexity as a function of repo size grows at something greater than linear rate. Code becomes tightly intertwined and the dependency graph becomes unwieldy, making refactoring a difficult proposition.

Eventually, the codebase always grows to a size that makes it impractical to replicate on each developers' machine, so you start having to implement workarounds, and you realize that just keeping basic build operations working requires a substantial team to support. For the big players, they can eat the cost no problem, but it's still not money well spent and certainly not advisable for orgs without that level of resourcing.

> Code becomes tightly intertwined and the dependency graph becomes unwieldy

One way to combat this is to use a build system with a notion of visibility, https://bazel.build/concepts/visibility

I think you might be over-extrapolating from your experience, but it's true going off the well-trodden road leads to surprises. And not that many smaller companies know the ins and outs of monorepos.

I have... as mentioned, worked in one of the large tech companies with a monorepo, and they all use fairly similar tooling.

In practice, it doesn't solve the problem because most people default to making everything public. Nominally this avoids duplicating logic, but may result in more stuff being pulled in than was really necessary. These days, when NodeJS developers discuss how bad things have gotten with dependency management, I just laugh.

How would you handle versioning in a monorepo? Just a new directory for major versions?

If we have multiple teams you can’t really refactor other teams code and sometimes you need to do breaking changes thus I imagine that some versioning must exist.

That's a great question. What I've seen is versioning de emphasized. Good unit & integration test coverage to prevent refactor breakage. And when you make a large change that affects consumers of your library, you find a way to do it incrementally. E.g. instead of changing a function's parameters, you can create a second function, migrate everyone over, then delete the old one. And of course, you would not ordinarily modify another teams code without first getting their LGTM.

A big part is engineering culture. At Google having multiple versions of a thing is an exceptional situation. But it's also acceptable (to an extent) to refactor other peoples' code and send them PRs. If you need to make a significant change, you do it in phases that allow you to maintain compatibility until you or someone else is able to migrate usages to the new ways.

I love the concept of monorepos. With the proper tooling and support I think they can be force multipliers for high functioning engineering organizations. However almost none of the publicly available tooling for CI/CD, IDEs, Source code hosting, or any of the other force multipliers out there support it properly, so for most organizations I think it's a wash.

It's a shame really since if they did support monorepos better or even just stop assuming too much about the repository then a whole lot of things could be unlocked for engineering teams. As it stands though going the monorepo route means you'll be building a lot of bespoke tooling to support it.

> With the proper tooling and support

Could you elaborate/provide some examples?

In a monorepo you usually only want to build portions of the codebase at any given time in your CI/CD. Most of the CI/CD tooling out there are accidentally hostile to this.

As just one example it is often difficult to impossible to rebuild your code when a dependency changes. Build systems often don't expose this information usefully, so you'll find yourself either hand rolling something that looks at git changes and tries to infer what should be rebuilt and what shouldn't. Or you just rebuild everything in the entire repo every night. Or you can adopt Bazel everywhere which can be a hard sell in many companies but does give you better tools.

Another example: Most source code hosting products don't give good support for path based permissions which you need in order to ensure that folks with the right context review code before it is merged in. Which pushes you to segregate your repositories along ownership and responsibility lines.

Google is the main example, their monorepo setup works and scales really well. It is basically a remote filesystem directory that automatically checks out the files you need to build your project and not other files.

Why would that be needed except for huge codebases at megacorps?

no one needs this except maybe google. even then if they started today with git maybe they wouldn't have used a monorepo.

same as pointing to netflix and arguing you need 100+ microservices. no one needs this, maybe except netflix

Using a monorepo was a deliberate choice at Google. They modified their source control solution to support a monorepo every time it failed to scale sufficiently. If they had started with Git those same people would probably have modified Git to make it work as a monorepo. There are specific economies of scale Google gets with a monorepo that help it keep up with the sheer amount of cruft and technical debt they generate everyday.

Google writes a lot of code and ensuring that security fixes get applied or legacy deprecated APIs get removed is for some people a full time job. Having a monorepo makes that job actually achievable rather than impossible.

Source: I worked there and was familiar with many of the engineers working on the tooling for the monorepo.

Are monorepos cool again?

Dunno, all my software development experience is in monorepos (Rails, Gamedev). Or simply one-off scripts...

From single PHP file to cover both back-end & front-end to multiple layers & repos to monorepo. Another full circle in SDLC.

I've worked at companies with monorepos and it pushed me pretty hard in the opposite direction. My philosophy has been to go through the packaging and dependency process early and often, and split off as many things into their open repo as possible.

Any specific reason you came to that conclusion when working with monorepos?

I'm a bit scatterbrained on this topic, maybe I should sit down and try to organize my thoughts. Every company I've been at that has used monorepos has fallen into a hell of dependencies where changes break things unpredictably. The other big issue I've seen is companies with monorepos ended up not trusting their engineers. Everyone being able to see everything results in lowest common denominator code, and style fights that would never matter without one big repo.

Ours is essentially a dumping ground, a disorganized network share. Oh, how great it would be to make the problem smaller.

Reply on news.ycombinator.com