So instead of cleaning up the internal dependencies, they decided to rewrite Mercurial. That is the kind of thing Facebook likes to do: for example, when PHP got too slow, they wrote a PHP compiler....
The scale of these codebases is way outside what most people have experienced and it might behoove people to realize that a lot of conventional wisdom might not apply. E.g. last time I checked, Google's codebase was >2Billion lines of code.
Google has 30,000 engineers working on a "monolithic" codebase and remain relatively productive despite (or perhaps because of it). There are whole sets of different problems at this scale.
Thus far I've preferred the mono-repo mostly for dependency management reasons. Whether you have a lot of internal dependencies or external dependencies, you get similar benefits: - All the changes to internal dependencies are in your revision history, across the company. Shared internal library upgrades are picked up quickly and propagate throughout applications with low delay.
- It's easier to have shared versions of external dependencies "automatically" instead of establishing policies that need a human to enforce. This makes it easier to roll out new versions and bug fixes.
- It's easier to do system-wide improvements in code quality. Replacing common bad code with better implementations is something I've seen across code bases at both Google and Twitter and they have been beneficial.
I think nobody should be trying to implement a system like this if you've got fewer than ~600 SW engineers, though. Small groups don't have as much drift or system-wide refactorings that give you benefit.
Package management is simple, we push common packages into sonatype nexus with semantic versioning.
Personally I prefer strong versioning and allowing teams to upgrade common libraries at their own pace, but we've had a few times where we needed to quickly upgrade every repository (i.e., a security patch).
You have a few options:
1. You can unpublish the old dependency, breaking all builds until they upgrade. (Frowned upon)
2. You can write a script that identifies all repository owners that depend on you and send out a upgrade by X date email.
3. You can script pull request creation and submit hundreds of pulls to upgrade the dependency.
So far, things have worked out well, and I never want to go back to the monorepo.
As a "one-man-team" who uses at least 15 different repos, it's hard for me to imagine how a massive company would manage things within a single repo and no package management.
Also, aren't any of those monorepo companies concerned that a single rogue employee or stolen laptop could leak their entire source code?
I can only guess I'm misunderstanding what is meant by monorepo and how they're used...
Personally, I also favor a single repo. You manage it the way you manage separate packages: with organization and some discipline. The magic is that command line tools like grep, sed & awk--and static analysis tools for refactoring--work really well. You can change a method signature and it just works. I've been part of monolith-breaking before (most recently at Trulia) and it definitely adds friction to the development process to work across such a rich graph of package dependencies.
* Compilation is done centrally, you code against a mock or only the interface and the submit the code for test and final build.
* Or only libraries are supplied, possibly ofuscated.
There are dependency management tools that help enforce public/private code on a wider scale and that help the build tools make sense of it all. There are also ownership tools that say what people and teams are qualified to review code in certain directories. The config files for all these tools are checked into the repository.
There's no versioning though. If you want to change an internal API you just update it and all the callers at once: patches in source control are already atomic. For truly massive changes (more code than most companies have) this gets too unwieldy and there are special tools and strategies people use.
This means you can't have a project depending on an out of date (internal) library. Without that requirement, you don't have the situation where different libraries need to be synced to different versions. And without needing to sync different things differently, you can get away with just one repository.
I worked at Amazon previously, which has world-class tools for dealing with versioned libraries in bulk, and Google's approach is vastly better. You spend less time worrying about breaking other people's dependencies, and you don't have someone spending a day fixing libraries every couple of weeks.
FBShipIt has been primarily designed for branches with linear histories; in particular, it does not understand merge commits.
Even with stolen FB code without dedicated infrastructure you still cannot do anything.
"Why didn't we think of this!!!?"
Maybe it's because these unknowns can't fathom working with such tightly wound systems that after all the code reviews are done your changes are irrelevant and need to be updated again and go through another review. In before straw man, but other than pointing out day to day issues, maybe you should try to understand the argument against trying to fit everything all in one place. Do you really want to be the maintainer of all those third party libraries you imported? Do you not allow third party and adopt "not invented here"?
Try again
Our code base has grown organically and its internal dependencies are very complex. We could have spent a lot of time making it more modular in a way that would be friendly to a source control tool, but there are a number of benefits to using a single repository. Even at our current scale, we often make large changes throughout our code base, and having a single repository is useful for continuous modernization. Splitting it up would make large, atomic refactorings more difficult. On top of that, the idea that the scaling constraints of our source control system should dictate our code structure just doesn't sit well with us.
I read it, but none of those things make me think better of their code.
Which works great until the day you wake up to realize your code base has become an incomprehensible mess and your engineering systems are outdated and your competitors are shipping better software at a faster pace.
I'm not really disagreeing, I'm just saying a balance must be struck. Really hate companies that view anything but feature work as a loss.
http://danluu.com/monorepo/ is a good explanation of why that's a pretty foolish take.