Hacker News Reader

Comment by przmk | Hacker News Reader

At my current $dayjob, there is a backend that is split into ~11 git repos which results in a single feature being split among 4-5 merge requests and it's very annoying. We're about to begin evaluating monorepos to group them all (among other projects). What would the alternative to a monorepo be in this case, knowing that we can't bundle the repos together?

View on HN

Is a mono repo the answer, or is the real problem you just have a bad repo split.

I can't answer that question, and there are reasons to go monorepo anyway. However if your problem is a bad polyrepo split going to monorepo is the obvious answer, but it isn't the only answer. Monorepo and polyrepo each have very significant problems (see the article for monorepo problems) that are unique to that setup. You have to choose what set of problems to live with and mitigate them as best you can.

"11 repos with 4-5 merge requests" doesn't sound like Google-level, so I don't see why a monorepo wouldn't work without much work.

The general rule is that things should be versioned together that change together. Separate repositories should be thought of similarly to separately versioned libraries. Dependencies between repositories should have stable interfaces. Design decisions that are likely to change should be encapsulated within a module, so that these decisions are hidden from other modules (a seminal paper about that is [0]). These considerations should guide any split into separate repositories.

[0] https://wstomv.win.tue.nl/edu/2ip30/references/criteria_for_...

Yup, at work I have a few projects split across several repos in like four languages. A completely new feature implemented across the whole stack involves PRs in up to 8 different repos. Potentially more.

To be totally honest, yes this is an unbelievable pain in the ass, but I much prefer the strict isolation. Having worked with (much, much) smaller monorepos, I find the temptation to put code anywhere it fits too much, and things quickly get sloppy. With isolated repos, my brain much more clearly understands the boundaries and separation of concerns.

Then again, this results in a lot of code duplication that is not trivial to resolve. Submodules help to a degree, but when the codebase is this diverse, you're gonna have to copy some code somewhere.

I view it sort of like the split between inheritance and composition. You can either inherit code from the entire monorepo, or build projects from component submodules plugged together. I much prefer the latter solution, but clearly the former works for some people.

At mine we ended up with two very comparable webapp products due to an acquisition.

One is built as a monorepo and we have a shared dev server where each user can run their own copies in a home directory.

The other is built as a collection of Docker containers that devs run locally. Nobody from the monorepo team likes dealing with it. Resyncing requires a much more elaborate Git process than a single fetch and pull. A simple task can spawn five merge requests to digest. We have loads of extra effort just making the QA team is in the same place as devs.

If nothing else, there's huge simplification from "I can access your copy of the code base and see the same error you're seeing" without trying to screenshare or remote-desktop.

I think you might just have a badly architected backend. get rid of your microservices first and then we'll see how you're feeling

As an asside, I've found IntelliJ very helpful in this situation as it can load many repos into one project then doing commits / pushes / branches etc across various repos at the same time just seemed to work the way I wanted without much thinking about it.

Do these 11 repos end up in separate binaries?

Because it sounds like you just need flag based feature releases.

It ends up with 6 deployables that are coupled together (let's say micro-services). There are surely better ways to structure the project but our CI/CD pipeline doesn't allow us to do so and it is not handled by our team anyway. I haven't seen any good way to make my life easier for merges, tech reviews, deployments, etc…

use git subtree - first to concatenate the minor repos into one major repo, and then subtree split from that point forward to publish subtree folders back to the minor repos, if needed (e.g. open source projects to github). works for us with about 8 minor repos, eliminated submodule pain entirely. only the delivery lead has to even know the minor repos exist.

I have already briefly looked at git-subtree. From what I can gather, it doesn't help much with my use-case. You still need to manually pull from each subtree and push branches individually to each project. The end result is still 4-5 merge requests to handle on Gitlab for a single new feature.

I might have missed something.

I believe dustingetz is suggesting making a monorepo for the code itself, but copying the subdirectories of the main repo into subrepos to solve your CI issues.

This means that developers have a monorepo for day to day work, but the CI/CD issues are isolated in their own separate repos, and can be handled separately.

Dunno if that's 100% of what they mean but it seems to be a solution to what you describe in another message ("our CI/CD pipeline doesn't allow us to do so and it is not handled by our team anyway")

Only 11 repos? I am at 76 repos for one backend. lol It's madness.

Reply on news.ycombinator.com