Hacker News Reader

Comment by anon9001 | Hacker News Reader

I'm glad you're having a good experience now, and git as a monorepo will work fine at smaller scales, but you will outgrow it at some point.

When you do, you have two choices. You can either commit to the monorepo direction and start using non-standard tooling that sacrifices decentralization, or you can break up your repo into smaller manageable repos.

I don't have any problem with small organizations throwing everything into one git repo because it's convenient.

My objection is that when you eventually do hit the limits of git, will you choose to break the fundamentals of git decentralization as a workaround? Or will you break up the repo into a couple of other repos with specific purposes?

I don't like that GitHub makes money by encouraging people to make the wrong choice at that juncture.

View on HN

> I'm glad you're having a good experience now, and git as a monorepo will work fine at smaller scales, but you will outgrow it at some point.

I would say the opposite. A lot of companies are fine with independent teams using their own versions of dependencies and their own versions of core code, but at some point that becomes unmanageable and you need to start using a common set of dependencies and the same version of base frameworks to reduce the complexity. That means pushing a patch to a framework means all the teams are upgraded. Monorepos are the most common solution to enforce that behavior.

Look, this is all dealing with the problem of coordination in large teams. Different organizations have different capacities for coordination, and so it's like squeezing a balloon -- yes, you want more agility to pick your own deps but then the cost of that is dealing with so much complexity when you need to push a fix to a commonly used framework or when a CVE is found in a widely used dep and needs to be updated by 1000 different teams all manually.

There is no "right" way. It's just something organizations have to struggle with because it's going to be costly no matter what, and all that matters is what type of cost your org is most easily able to bear. That will decide whether you use a monorepo or a bunch of independent repos, whether you go for microservices or a monolith, and most companies will do some mix of all of the above.

> Monorepos are the most common solution to enforce that behavior.

Yes. This is very accurate and also the problem. Monorepos are being used as a political tool to change behavior, but the problem is that it has severe technical implications.

> There is no "right" way.

With git, there is a "wrong" way, and that's not separating your project into different repos. It causes real world technical problems, otherwise we wouldn't have this article posted in the first place.

> It's just something organizations have struggle with because it's going to be costly no matter what, and all that matters is what type of cost your org is most easily able to bear.

It's not a coin toss whether monorepos will have better or worse support from all standard git tooling. It will be worse every time.

The amount of tooling required to enforce dependency upgrades, code styles, security checks, etc across many repos is significantly less than the amount of tooling required to successfully use a monorepo.

If you want to play right and wrong, I will say that noe it’s the right way since there is support for sparse checkouts in git.

This isn’t a useful game to play.

> The amount of tooling required to enforce dependency upgrades, code styles, security checks, etc across many repos is significantly less than the amount of tooling required to successfully use a monorepo.

This is completely false. You only need extra tooling on monorepo for gigantic scale. For 90% or maybe even 99% of companies out there using version control, plain git and a "bunch of folders" is all you need, honest to god. This way you can reap most benefits and very few downsides. Detecting changes in folders is pretty easy, as most CIs/CDs have it built-in. Ff not it's easier to program a script than setup multirepo tooling.

Being on a known-good version of a dependency is tech debt that I want my team to pay down when we have the slack. Alpha testing every commit of everything in prod is not our job. Google3 had company-wide build breaks so often (and hours of retest backlog) that teams had a “build cop” whose job was to urgently complain about them. Eventually they had to pick a set of active heavily-used libraries (e.g., bigtable) and set up the build system so most consumers would not depend on them at the very latest changelist.

If you are in an enterprise setting, you don't need decentralized version control.

So, yea, for companies, monorepos are a no brainer in a lot of ways.

For open source, separate repos makes more sense.

To expand on corporate monorepos, if you can still set up access control (e.g., code owners to review additions by domain) and code visibility (so there isn't _unlimited_ code sharing), then I can't think of a reason to not use monorepos.

When I hit the limits of git then I will worry about it.

One of our tasks when building the monorepo was proving it was possible to split it again. It was trivial and we have tools to help us avoid complexity.

We’re not using Github so that part doesn’t apply to me.

Also, nice of you to assume we’ll get to Google scale, but thanks to the monorepo, I was able to make a few pull requests reducing duplication and reducing line count of app by thousands ever since. So I really don’t see us getting into Google scale anytime soon. We're downsizing.

I also find it ironic that you’re accusing people of “copying Google” in a parent post but you’re the one assuming that everyone will hit Google limits…

If you ever do hit a git limit where it's no longer comfortable to keep the whole repo on each developer machine, I would encourage you to split up the repo into separate project-based repos rather than switching to Microsoft's git fork.

As a best practice, there's a reason that Linus started git in a separate repo, rather than as part of the Linux project. The reason is that if you put too many projects into one git repo, and it gets too large, you do eventually hit a scale where it becomes a problem.

A very simple way to mitigate that is to keep each project in its own repo, which you can easily do once you start hitting git scale problems.

Thankfully, one of the original git use cases was to decompose huge svn repos into smaller git repos, so the tooling required is already built in.

> I honestly find it ironic that you’re accusing people of “copying google” in a parent post but you’re the one assuming that everyone will hit Google limits…

I think you got the wrong take there. I'm saying that Google's monorepo approach is only valid because they invested so heavily into building custom tooling to handle it. We don't have access to those tools and therefore shouldn't use their monorepo approach.

If you're going to use git, you're going to have the most success using it as intended, which is some logical separation of "one repo per project" where "project" doesn't grow too out of hand. The Linux kernel could be thought of as a large project that git still handles just fine.

Tragically, I think if Google did opensource their internal vcs and monorepo tooling, they would immediately displace git as the dominant vcs and we would regress back to trunk-based development.

> As a best practice, there's a reason that Linus started git in a separate repo, rather than as part of the Linux project. The reason is that if you put too many projects into one git repo, and it gets too large, you do eventually hit a scale where it becomes a problem.

Nope. The reason they are in a different repositories is that they are completely different things, with virtually zero code shared between them. There is no point in putting them in a single repository. The teams are different, the process is different, the release cycle is different.

The reason people like me use monorepos is not because we mistakenly believe "each company needs only one repo". It's because there is a tangible and measurable advantage in grouping some things together. That's it.

> If you're going to use git, you're going to have the most success using it as intended, which is some logical separation of "one repo per project" where "project" doesn't grow too out of hand.

No, that's patently false and absurd. I had the experience of using one repo per project and it was a horrible experience full of bureaucracy, and I know other people with the same experience. A git monorepo is working fine for my company and has been for years. I know other companies who also feel the same, and no, the "sky is not falling" as you seem to believe.

Claiming that "there can be only valid method, and it's mine" is bullshit. Engineering is not about silver bullets.

> Tragically, I think if Google did opensource their internal vcs and monorepo tooling, they would immediately displace git as the dominant vcs and we would regress back to trunk-based development.

You keep forgetting that not every company operates at the Google level. Git is more than fine for 90% of the cases out there. There is more to tech than FAANG.

> If you ever do hit a git limit where it's no longer comfortable to keep the whole repo on each developer machine, I would encourage you to split up the repo into separate project-based repos rather than switching to Microsoft's git fork.

Sorry to be blunt, but I'm not taking advice from an internet stranger with opinions grounded purely on dogma, and with an axe to grind. I will carefully study my options and take a decision based on my needs. There are no silver bullets in engineering.

> you will outgrow it at some point

Given that Google and Microsoft use monorepos that seems unlikely!

Google had to build an internal version control system as an alternative to git and perforce to support their monorepo.

Microsoft forked git and layered their own file system on top of it to support a centralized git workflow so that they could have a monorepo.

Having had used both, Google's implementation is IMO the superior version of monorepo. Really, Google's Engineering Systems are just better than anything that I have ever used anywhere else.

This is exactly as I'd expect.

If you want a centralized, trunk-based version control, don't use git.

It's funny how each company decides to solve these problems.

Google called in the computer scientists and designed a better centralized vcs for their purposes. Good on them. It'd be great if they open sourced it. So typical of Google to invent their own thing and keep it private.

Microsoft took the most popular vcs (git), and inserted a shim layer to make it compatible with their use case. How expected that Microsoft would build a compatibility shim that attempts to hide complexity from the end user.

Meanwhile, Linux and Git are plugging along just fine, in their own separate repos, even though many people work on both projects.

> So typical of Google to invent their own thing and keep it private.

Yeah like their build system... Bazel, that's completely closed source.

Google did not design a VCS from scratch, they adopted Perforce and started replacing its backend as they hit scaling limits. They eventually added some kind of copy-on-write FUSE layer that looks like your workspace contains the entire monorepo, which was pretty cool.

It would be hard to open source if its dependencies only exist on Borg. I think that was a problem with Blaze (now Bazel).

Your logic is circular.

    No one should work on monorepos because...
    monorepos are bad because...
    git can't easily handle them and we shouldn't fix that because...
    No one should work on monorepos...

Clearly there are reasons people like monorepos and it makes sense to update git to support the workflow.

That isn't circular. The conclusion should be that git, a decentralized vcs, should not take on changes to make it a centralized vcs.

If you think that git needs to be "fixed" or "updated" to support a centralized vcs server to do partial updates over the network, then I think you've missed the point of git.

Sparse index has literally nothing to do with VFS for Git. The sparse index features are in microsoft/git because it's a convenient, deployed-at-scale test-bed before upstreaming into git/git.

Reply on news.ycombinator.com