Comment by hpaavola | Hacker News Reader

Comment by hpaavola | Hacker News Reader

With my current client we decided to go with multiple repositories and came to regret that decision.

The product family we are developing contains website, two mobile apps (iOS, Android), three PC applications (OS X, Windows and one legacy application) and software for embedded devices.

Each product lives in it's own repository, most repositories use one shared component as a submodule and many product share common platform, which is used as a submodoule and products build on top of that. Test automation core sits its own repo. I built and maintain that test automation core and it's a pain.

Each product repository has it's own functional tests that use the test automation core to make the tests actually do something. So when ever I make changes to the test automation core, I need to branch each product and use the feature branch from my core in it. Then run all the tests in each repo and see that I did not break backwards compatibility. If I do break it, then I need to fix it through pull requests to possibly multiple teams.

I'm not the greatest git wizard in the world, so maybe someone else could easily maintain good image of the whole mess in their head, but for me this is a pain. And everyone else who maintains a shared component shares my pain.

Monolithic repo would not magically make all the pain in the world to disappear, but it would be so much more easier to just have one repo. That way I would need to only branch and PR once.

For repos to be independent, they need to have a defined interface, be able to test themselves against that interface as well as the expected behavior, and specify interdependencies with versioning (semver helps reduce effort on this point).

You can run integration tests in the dependent project across the interface of the dependency project, but tests in the dependency project should find 99% of breaking changes in the dependency.

So you can do whatever you want in any project, then test those changes. If your tests point out you've broken your contract with other projects, you decide to either solve that issue before releasing the change or cut a new major revision. Behavior that isn't covered by tests should basically be considered undefined, teams of dependency projects should contribute tests for things they want to rely on.

Either way the other project doesn't break -- it either updates to the new compatible version or stays behind on the last compatible release until it's been updated for the incompatible change. You should be confident having a robot automatically test projects with updated versions of dependencies and committing the change.

Of course, it's not always possible or desirable to have this degree of engineering around projects, but then the reality is that these aren't independent codebases, they are a single interdependent project with a single history and should be versioned/tested/released as such.

I understand your logic in theory, but not in practice. I maintain some 20 or so large applications, that all use a shared data access layer. That dal is safely cordoned off behind an interface layer, but I still have the same problem as the parent poster. If you make enough changes to any one of the subscribing applications, you will need to make an update to the DAL. If you update the DAL, you will need to make updates in the 19 other applications, and run the tests for all of them, if only to ensure they compile.

I could break the DAL further apart into 20 different pieces, one per application, but there is so much shared data access functionality between the applications, that it doesn't make sense.

There's no "logic in theory" and "logic in practice" there's just logic and people failing to apply it in practice.

If you can't come up with a small, stable interface between the the 20 applications and the shared component, then you don't have 20 applications + a shared component, you have one big app that is painfully maintained as separate repos across a large cross-sectional arbitrary boundary.

If the history of the project is so interdependent as to be singular (everything is changed in lockstep across repos, effectively a single history) why not have a monorepo? If versioning them under the pretense that they are independent modules costs you so much effort, why do you labor to do so?

Just to be clear, wouldn't the alternative with a monorepo still require that you go back and forth with multiple teams if the commit is not backwards compatible? It seems like the main complaint you have is that it's difficult to wrangle a number of related pull requests, so perhaps switching to something like gitcolony [1] for code reviews would help.

[1]: https://www.gitcolony.com/features

Never heard of gitcolony before. Looks interesting, thanks! That would probably solve one issue. Then there is another one.

I try my best to keep head of master branch in such a state that it can always be taken into use in all projects. Just last week one branch of one embedded device had slight API change. Nothing big, but backwards incompatible change.

I branched test automation core and made it work with the new API. All tests looked green and things looked nice. We agreed with the embedded device owner that we'll merge that change upwards soon. Soon like in one hour or so. I rushed and merged the test automation core changes to master.

At the same time I was working with another API change with one PC app. That branch was also looking good and those changes were merged upwards in both the test automation core and in the PC app.

Now my test automation core master head was compatible with everything else but one embedded device, the one with the small API change that looked good in all tests. For some reason business did not want that change to go live yet with the device, so now I had changes in my test automation core that made it incompatible with the released embedded device.

Yes, it was my mistake to rush with the merge. But because getting those changes upwards was two merges, one in the product itself and in the test automation core, it was possible to get those out of sync. If we had used monolithic repository, it would have been just one merge and such thing would not have been possible.

Sure, not a huge thing but still an annoyance I could live without.

Tests belong in the project they're testing. Test tooling belongs elsewhere. It's not clear what your setup is, but it sounds like the tests themselves live in another repo. That's bad. It's not bad to have a separate repo for your tooling/test runners. As you've just seen, new tests need to go out simultaneously with new code.

Yes, tests are with the code as are test resources that are product specific. In addition to those there are a lot of resources that are shared across all or most products that sit in the test automation core.

> Just to be clear, wouldn't the alternative with a monorepo still require that you go back and forth with multiple teams if the commit is not backwards compatible?

No. You just send one diff that changes all the team's code and update everything in lockstep, so at any point in your history, everything is compatible.

Instead of you going back and forth with multiple teams, you're bringing them together to comment on a change in one place. You synchronize on version control history instead of needing to wrangle multiple teams across multiple repositories, and you no longer need to deal with fallout for code change compatibility. You just make the change. Everywhere. In one shot.

You may have to get multiple teams to review your change before being allowed to commit it. And you have to run all their tests. If there is a problem the whole thing will typically get rolled back, which is a drag because then you have to fix the issue, run tests again and get approvals again.

So, in practice, for large-scale changes that affect many teams, we still try to break up large patches into multiple smaller steps, even working in a monorepo.

A single commit is nice for small patches that only affect a handful of teams, though.

Ugh. I inherited a project that was needlessly broken into two repos and it's been a constant source of irritation.

Reply on news.ycombinator.com