You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.
That has not been my experience at all. At a previous employer, we did exactly that with a multi-language library. In fairness, having multiple languages enforced fairly good directory structure in our single repo. But isn’t that the real point: good structure makes life easier, period. The thing is, going into a project you often don’t know what the right structures are yet. Creating a new repo for each component you think you need ossifies those choices, making it far more difficult to walk back on them later on (first because you may not even see the architectural mistake, second because the maintainers of that component will have an investment in its existence).
> You can always just put all your small repos into a big one.
In my experience, that’s harder, precisely because over time so much tooling has been built into each repo to manage builds, images, deployment &c.
I’ve worked in monorepos & I’ve worked in multrepos, and so far my experience has been that monorepos enable faster velocity and more-maintainable software. I’ve not (yet) worked at Google- or Facebook-scale, though, and I’m completely open to the idea that at that scale a team really does need lots and lots of repos, and tooling to stitch them all together.
If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git, that could lead to multiple root commits, but that's about it.
Most of the time they're just annoyed.
One side effect of every successful business are annoyed worker ants that are sick of dealing with growth problems. I've been there. I know how annoying it can be.
Personally I've found comfort in embracing the chaos and learning to manage it responsibly. No dogma. No absolutes. Know how to do monorepos well. Know how to do polyrepos well. Learn the pitfalls of both. Don't assume other people are stupid zealots.
> The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business
I'm not really sure how I should read this. Don't you use your repos to solve business problems? Why should that change because of the repo layout?
Why would you put in a symlink? You could just provide a path to the actual component and import it into your project.
> the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one
When you have multiple repos, you also have multiple versions and releases of things. Now every team has the following options:
1. Backport critical fixes to every version still in use (hard to scale)
2. Publish a deprecation policy, aggressively deprecate older releases, and ignore pleading and begging from any team that fails to upgrade (infeasible - there'll always be a team that can't upgrade at that moment for some reason)
You also have to solve the conflicting diamond dependency problem. This is when libfoo and libbar both depend on different versions of libbarfoodep. It's even more fun in Java because everything compiles and builds, but fails at runtime. So now you have to add rules and checks to your entire dependency tree - some package managers have this (Maven), others don't (npm IIRC).
Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.
I don't know anyone who has abused Maven or Cargo or Go like this. And I don't imagine Visual Studio Solutions for C# are used like this.
Is there an underlying disagreement based on JS/Ruby/Python scriptish coding (which creaks when a lot of developers work on it) vs C and C++ (which have astonishingly bad build system stories) vs big-iron languages that don't sweat when in a monorepo.
At my workplace, we've just been cleaning up a whole bunch of instances of exactly that anti-pattern. Except that it's obviously not symlinks (which require specific user rights on Windows), but links to external files in VS.
Same problem, though: They're easy to introduce and a pain to deal with later on.
I've seen that with polyrepos as well: The entire project would require you to clone the individual repos into a specific directory structure so that things would work (no, not even submodules).
It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for.
For any 3rd party libraries used by your projects you need to either ingest those projects into your monorepo and update forever. Or keep npm/maven/pip/gem/whatever around just for managing 3rd party dependencies (+ whatever system you use to front the main language package registries, because of course you're not talking directly to NPM/Maven central are you? What if they go down or do a leftpad?).
I think either system - monorepos or polyrepos - works fine; just pick one and stick with it. Monorepos will probably give you better velocity starting out. Past a certain size, which most software shops will never hit, the already-available tools lend themselves better to polyrepo. And more devs know polyrepo tools (eg. Jenkins) than the corresponding monorepo solution (eg. Bazel). Things might swing in favor of monorepo on the VCS side if Twitter/Google/FB ever open-source their stuff.
It's their job. If they actively don't want to do work then you probably made a hiring mistake somewhere. By that logic what DevOps really wants is to the company to shut down since then they'd have none of that tedious work to do.
Sure it is their job, but is isn't an easy job and there are many opportunities for things to go wrong. It might or might not be the right choice for you, but don't overlook how hard it is.
Note that the above applies for going in either direction.
If I was moving many small repos into a single mono repo then I'd do it one repo at a time. Presumably your small repos are independent entities so there's no reason to do a single massive switch. Transition each repo to the new build system inside the existing repo. Once that works then you can transition that repo into the mono-repo and tie together the build systems. No need to stop releases, no massive chance of everything failing, no weeks of debugging while the world is stopped, etc. Rinse and repeat until everything is moved over. Process becomes more optimized and less error prone with each repo that is moved over.
It's harder to make a business value case for this type of change - there are only vaguely worded promises of "improved developer velocity". Contrast that with a change that automates or makes faster some aspect of building and releasing - a professional release engineering team would be all over it because they can demonstrate value in that work.
In any company beyond 50 people, there are multiple engineering managers, directors or the VP of engineering that will need to back this initiative to make the release engineering team do it. It's really not as simple as "dump all the code in one repo". I'm speaking from experience.
Sure you can. The difficulty of doing so depends on many (many) factors. If your team does their job well then the costs won't be immense. It might be annoying, but not that hard.
Speaking in absolutes or platitudes solves nothing. Sometimes monorepos make sense. Sometimes polyrepos make sense. It's entirely dependent on what your company does.
E.g. if relying solely on a package manager to to keep coupled things in lock step, you need make sure that version numbers are kept up to date for every little change made to every library.
You can easily end up with a situation where someone in another team makes a small change but doesn't change the lib version number. That's a people issue but it does happen.
You can get round that by using a repo SHA but now you two things to keep up to date for every library.
Like wise you'll have to be diligent in versioning APIs. Anecdotally I've found it easier to keep things in lock step when in a single repo and using a single pull request for each story than I have where separate teams have to keep separate repos in sync.
Both work but the monorepo approach worked better for the projects I've worked on. It just lead to less moving parts and more repeatability when there's a single SHA to watch.
I also have been luck enough that I haven't worked on a project so large that we couldn't build a monorepo on a single machine with "normal" build tooling.