Monorepos: Please Don't (2019)
medium.com
medium.com
Moving to monorepo is often a win; also the name sucks, we have a a few "monorepos" at work, and I think it's the sweet spot. Rust and C firmware doesn't need to live with TypeScript frontend apps, but it's pretty wasteful to have apps of the same ecosystem unable to share any and all dependencies and utility code trivially.
(Where "trivially" means literally one line of code
import { bleargh } from 'hoge';
I get monorepos for SSR and the likes but why build and deploy an entire container to push a line of CSS?
Imagine adding a field to a table that must be reflected in the front end and some other micro service consumers. Coordination can be painful.
Each system has its pros and cons, you're going to need tooling either way. I've moved to mono or mega repo as the default and break out smaller pieces into dedicated repo as needed
Your monorepo sometimes approach is the polyrepo approach.
To me the ability to push changes independent of other pieces far outweighs the extra git commit and push. I've never had the need for any other tooling than a container to run in, git to manage code, and CI/CD for deploys. If there's a need for special tooling to manage the projects there's to much complexity for my taste and I remove the complexity instead.
We could look to deploy CSS in a different way to optimize the use case for updating a single CSS line, and deoptimize everything else.
Monorepos make sharing code easier. They don’t put any requirements on deployment. You can make good or bad decisions regarding deployment when using monorepos or multiple repos.
Instead of e.g., "blah blah blah a monorepo — not necessarily a literal monorepo in terms of having only one repo with every single project in it, but a repo in which several related projects are colocated, and that uses some kind of monorepo tooling like Nx to avoid redundant work — blah blah blah".
I have only worked in large codebases with monorepo, so I'm curious.
Jenkins creates all the protobuf definitions and pushes them to their dependent repos.
A better way would be with a package repository for them, and having it versioned; and running multiple versions of the backend.
This article is also 5 ish years old and I would hope some of the tooling for bigger orgs has started to make it downstream since then.
As parent comment mentioned, scaling welcomes teams to work on tooling.
A mix of Gitlab overall with some Gitolite to manage access to certain branches of the repo if needed seem to be one way to go.
The reality is that these are somewhat niche usecases for very high scale orgs and/or very high-scale developer workforces which can afford to reinvent the wheel. The way the vast majority of companies should go about developing their systems is not new at all, its just to pragmatically apply tried and tested technologies, and transition to new shiny things where there is a demonstrable need.
Worth saying that I'm definitely a fan of team-scoped monorepos though. Being able to automate building and testing across all my apps, update dependencies across all apps, and deploy with a single merge/pull request is great.
For example, the Linux kernel works with its tens of millions of lines just fine, even though ownership is heavily distributed among many subsystem maintainers. Checking out a branch or pulling in changes rarely takes more than a few seconds.
On the other hand, some organizations abuse their VCS, using it as a repository for large immutable files such as build artifacts, software packages or similar non-source archives. Orgs who let this happen are of course going to have a hard time. But that's hardly the monorepo's fault.
Once an org becomes large enough you need lots of custom tooling to make working across a huge codebase smoother, and that tooling is similar for mono and poly repos, but you need an additional metric ton of tooling to make monorepos work
And please pray you didn't decide to go with bazel for builds
My least favourite monorepo experience: I want to update my dependency for my tiny service for a security patch -> Oh, 400 other services depend on it and dozens of tests break when I try to update it -> this isn't worth my time I'll do something else instead
We were promised a world where every dependency was kept up to date by necessity but ended up in a world where all dependencies atrophied due to the increased difficulty to update them
If the API changed it's not a security patch... Why would a change that only fixes a security bug cause tests to fail?
Sounds like this is more a matter of needing to cherry-pick temporarily and then actually pushing for the codebase to update, probably by the security team.
Sometimes you need to work with other people, that might necessitate doing "ugly" things to get the job done.
I was always hoping that one of the big cloud providers would offer a monorepo and invest a lot in making tooling for it to actually be usable.
Why not? And what should one use instead?
In particular, it fostered a culture where accepting code changes from outside teams was normal (though obviously not always uncontentious), and prototyping those changes was pretty easy.
I've been using pantsbuild.org since I left and I think it hits a good sweet spot between blaze, which has a lot of overhead, and the rest of the world.
The biggest issue is deciding what to build across developers and their branches & products. You don't want to build the world on every commit, but you do want to ensure that you are building what the code would look like post-merge before the merge.
MonorepoS, in the plural?
It's not a negative thing as much as an opportunity to grow if we ask ourselves to learn and see it as an opportunity.
But I guess that's not a result of a monorepo inherently. It's just the result of poor engineering leadership.
Our team produces 4 libraries. The code for them along with test apps is spread across 7 repos. So when I want to make a core change (like updating the version of Gradle we use) it results in 7 different PRs that have to go in at the same time, and that everyone has to fetch in sync. It's pretty rare that a change only requires a single PR because of this.
Releases are a mess. To do a release we tag the latest commit in each of the repos with the same release number and use Jenkins to pull that tag from each repo and put it all together. But that's just like a suggestion, and it's not uncommon for a release to be based off of master. Good luck doing the forensics to figure out the state of everything for a release.
The best part is that each repo needs to be in a specific folder on disk because some of them look for the others via hard-coded paths.
There is one repo that needs to be separate, because it's public. But I've been pushing hard to get most of the repos pushed together into a single one just to add some sanity to my PRs.