Hacker News Reader

Comment by gilfaethwy | Hacker News Reader

> By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.

Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.

View on HN

> developers are now checking out a 50GB repo in its entirety periodically

It would, of course, be easy to say "But that's not the fault of monorepos! Clearly, 50GB is a nonsensical amount of source code, and clearly someone committed something they shouldn't have in the past."

But also, with monorepos, the probability of that happening to the repo you use the most is the cumulative sum of all of its projects, since more people's potential git mistakes now happen in the same place.

For the sake of comparison, nixpkgs is 4.5GB right now. It hosts ~140.000 packages, ~12.000 open PRs, one million commits, yadda. So when you reach ten times that size with, presumably, less traffic, I would consider cleaning up the history.

Polyrepos can definitely work; I use them extensively in open-source. I also wouldn't be doing that without Nix flakes all over to chain them together. And even then, I have constant drift because one repo is pinning an old version of another repo, and once I bump the pin, things break lazily.

The biggest benefit, by far, you get from monorepos, is eager evaluation of your dependencies.

I'm on the other side of this problem, with a company that went multi repo for bad reasons (political, not technical) and I would give you serious money if you could solve my problems by just forcing me to check out 50gb every now and then...

Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.

Long story short... I will take a bad monorepo over bad multirepo any day of the week.

I suspect that your hours of effort to get everything up to date would exist in a monorevel, too. It would exist in a different form, and so it would be harder to measure, but a large part of the work has to be done either way. It's just that certain parts of the work become very visible.

Yeah, compared with the maintenance effort of running around to 100 different repos, keeping everything in sync and the deps all updated, I'd always take the pain of 50GB checkouts.

It looks like these days a combination of git-scalar, git-lfs, and bazel will work for a small 50GB monorepo like that. The largest cost would be porting over the existing build system to bazel and then integrating it with CI. Once that is done, though, the monorepo will just scale, and all the constant ongoing cost of multirepo sync will be gone.

Reply on news.ycombinator.com