Hacker News Reader

My current team managed to break a single "component" out into a separate repository. Then that repository broke into two, then those broke into other repositories, until we've eventually have around 10 or so different repositories that we work on every day.

An average change touches 4 of them, and touching one of them triggers on average releases on 2 or 3 of them. Even building these locally is super tedious, because we don't have any automation in place (not formally plan to) for chain building these locally.

This is a nightmare scenario for myself. A simple change can require 4 pull requests and reviews, half a day to test and a couple hours to release.

Yet my team keeps identifying small pieces that can be conceptually separated from the rest of the functionality, even if they are heavily coupled, and makes new repos for these!

I’ve come to the conclusion that an organisation should ideally have no more than one primary repo, with maybe a handful of ancillary repos for stuff that really doesn’t make sense in the primary. What does ‘organisation’ mean there? Well, it could mean a company, or a team, or a division. Just as software conforms to organisational structure (Conway’s Law), so too should repo structure.

Once you start having lots of peer repos being worked on within the same organisation on a daily basis, you know that you’ve partitioned far too far, and you need to roll back.

Otherwise one ends up in exactly the position you’re in. The ultimate slippery-slope end-state would be hilariously bad: a repo for each ASCII character, with repos for each word or symbol constructed out of those characters, with repos for each function constructed out of those words & symbols, with repos for each module constructed out of those functions, with repos for each system constructed out of those modules, with any change requiring a massive, intricate, failure-prone dance in order to update anything, all while patting oneself on the back about how one has avoided complexity.

Noöne sane would argue for that situation, and yet I’ve seen smart people argue that requiring coördinated changes to half a dozen repos is fine & dandy.

even if they are heavily coupled,

So don't use polyrepos for heavily coupled projects, then. Or even better...

... try to avoid heavy coupling in the first place.

Unfortunately, these debates tend to be of the bikeshed variety.

Q: Why are we debating the merits of mono-repos over poly-repos?

A: Because it's managing dependencies is really hard and needs expertise.

It's an interesting social problem in how you manage those project / library / repository boundaries. On the flipside, though, it's been well documented that among many of the major monorepos those boundaries still exist, they just become far more opaque because no one has to track them. You find the weird gatekeepers in the dark that spring out only when you get late in your code review process because you touched "their" file and they got an automated notice from a hidden rules engine in your CI process you didn't even realize existed.

In the polyrepo case those boundaries have to be made explicit (otherwise no one gets anything done) and those owners should be easily visible. You may not like the friction they sometimes bring to the table, but at least it won't be a surprise.

http://wiki.c2.com/?ConwaysLaw "Conway's Law" is something like "organisation of code will match the organisation of people". It's a neat description.

I think it's more common to merge or split modules and classes than repositories. I wonder if there'd be less tension if repos and teams were 1:1 though.

> I wonder if there'd be less tension if repos and teams were 1:1 though

Anecdotally, yes I think it helps a lot. I was once part of an organization for which each "team" having a repo is the only thing that prevented violence :-)

I've also seen arbitrary separations of repos because 2 people didn't get along and couldn't work together.

Can a monorepo support module- or subdirectory-level ownership controls? Or do teams using a monorepo just do without them?

Partially answering my own question: SVN, recommended in a prior comment [0], supports path-based authorization [1]. But what about teams using another version control system?

[0] https://news.ycombinator.com/item?id=18810313

[1] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.serverc...

SVN allows for you to create mutliple repos within a repo. (That's probably why the path based auth works).

Git has the idea of sub-modules, but they're really just filters. (They're in the same repo). So ultimately, you don't have that kind of control.

Git submodules are not in the same repo, they are a link from one repo to another, and you need to push to both if you make a change to the submodule. Maybe you're thinking of subtrees? I've never used those.

With Phabricator, yes, you can setup herald rules that stops a merge from happening if a file has changed in a specific subdir.

We use service owners, so when a change spans multiple services, they are all added automatically as blocking reviewers.

Piper, Google’s implementation of monorepo, has that, and it is very important and widely used.

If I remember correctly, Gitlab has introduced some sort of ownership control where you can say who owns what directories for things like approving merge requests that affect those directories.

Github has this feature[1], which we use extensively in our monorepo.

[1] https://help.github.com/articles/about-codeowners/

Hey, did you mean of assigning approvers based on code owners [1]? You can find more info about Code Owners and syntax at the documentation [2].

[1] https://gitlab.com/gitlab-org/gitlab-ee/issues/1012 [2] https://docs.gitlab.com/ee/user/project/code_owners.html

google3 uses the same model that Chromium does, see an example here: https://github.com/chromium/chromium/blob/master/chromeos/OW...

In Google, this is pretty explicit with a plaintext OWNERS file in the directory. Internal IDEs not only have an understanding of that, but can automatically suggest a minimal set of reviewers in a possibly close time zone and not out of office.

For my clients, I use an open source monorepo submodule inside the client's proprietary monorepo. I can maintain the organizations' software while sharing common code.

So (mono)repos are composable.

OctoLinker really helps when browsing a polyrepo on Github:

https://github.com/OctoLinker/OctoLinker

You can just click the import [project] name and it will switch to the repo.

It's very much possible to make changes to internal libraries used all over the place, but it does require versioning to be something that people think about, and a mechanism by which those libraries aren't just pulled from source control to depend on them. Once you've got some sort of dependency management such as an internal gem/npm/whatever source you can treat those internal dependencies the same as you'd treat external ones, instead of having to somehow coordinate a release of absolutely everything in one go.

That's not really that different in a monorepo since you often need reviews from the same number of people anyway.

I once had to wait for 9 months to get a complex change through in a monorepo setting because of all the people involved, the number of stuff it touched and the fact that everything was constantly in flux so I spent half my time tracking changes. I'm not saying it would have been faster in a polyrepo. I'm saying that complex changes are complex regardless of how the source is organized.

I do however think that polyrepos forces you to be more disciplined and that it is easier to slip up in a polyrepo and turn a blind eye to tighter couplings.

The multi-repository code review is an interesting concept. Here at RhodeCode we're actually working on such solution to implement. This is in first to solve our internal problem of release code-review spanning usually two projects at once.

This is a hard and complex problem. Especially how to make code-review not too messy if you target 5-8 repos at once.