Hacker News Reader

Comment by mort96 | Hacker News Reader

I've gone through such a migration. Huge Go code base distributed across a lot of git repositories had to be moved to a different git host.

It was horrible. A team of people spent weeks. Different projects depended on different versions of the same internal libraries, we had to create a branch for each depended-on commit and make a version of that commit with the new URLs. The expressed goal was to end up with a system that's "exactly the same" as the old, just on a new host. Changing which version of libraries projects depends on would introduce unnecessary risk.

And the result is a code base where bisects are broken and where it's impossible to build an old version of any of the code without a ton of work.

View on HN

> Different projects depended on different versions of the same internal libraries, we had to...

This is one of the main motivations for using a monorepo with all third party dependencies imported all the way in, and only one version for everything.

Of course it causes a whole bunch of other problems and is kinda expensive to scale, so most places won't do it.

Isn’t this exactly why the Go team recommended vendoring deps for years and years before go modules came up? Even now it takes one simple command to vendor them.

Well vendoring is a technical solution of the same caliber as adding module aliases to the go.mod. By that I mean, sure, it keeps the code compiling right now, but all your source files still contain references to abandoned infrastructure. It changes when you have to do the work, but unless you want your code to reference abandoned infrastructure forever, you still need to do the work.

Google runs a monorepo which makes that kind of organization possible. Most companies are going to have a mismatch of teams using incompatible library versions.

Pick your poison. Each approach has some seriously negative trade offs in the extremes.

Similar pain as upgrading a module to a new major version (eg: v1 -> v2) [1], but at scale, because of modules dependencies: every module in the domain are affected at once as they must change namespace, like if everyone changed its major version.

https://go.dev/doc/modules/major-version

I think is usually a symptom of how hard it is for the organization to do things, the technical side of this is not actually that hard.

I say this because I’ve gone through a few of these, some with monorepos (the easiest, search and replace and you’re done), some not (yes the hardest but you can just vendor).

At least in the situations I’ve faced this was genuinely not horrible beyond just the organization itself being complicated about the solution.

How do you propose it's solved in an easier way, given the constraints that 1) you don't want references to old infrastructure in your code and 2) you want to do the move without changing how the code works? Is there some secret trick we missed which would've made this much easier?

Everything is on A. All code refers to A.

Clone Server A -> Server B. All code still refers to A.

Update code to refer to B

You can now EOL Server A, and B becomes the new canonical reference.

I think the key is simply that you can keep both A + B running during the migration, so you just need to be able to do a code freeze for the duration of the migration. And a single person can easily do this migration with a few python scripts and an hour.

Bonus points: code freeze guarantees your #2 - no changes to how the code works.

Of course, I'd assume that most complaints come from situations where this "secret trick" isn't viable

The "update code to refer to B" step is a massive undertaking. My whole original comment was about that one step.

I don't understand. If B already exists and is working, isn't this just Ctrl+F, compile, push? I'm utterly unfamiliar with the "Go" language

No, it's not. Here's a comment where I described a concrete hypothetical scenario: https://news.ycombinator.com/item?id=49871032

You vendor the deps

> And the result is a code base where bisects are broken and where it's impossible to build an old version of any of the code without a ton of work.

That makes no sense? As long as you’re using go modules, you can just host an internal go proxy for internal modules similar to proxy.golang.org to archive the old versions, and they won’t depend on the old git host.

I don't know enough about git but shouldn't you be able to take all your git data (refs, blobs etc) and import it to your destination host and have this work without issues? It might be a lot of data sure but the infra is surely cheaper than multiple weeks of engineering effort

Moving the repositories over was no problem at all! But once all the data is on the new hosts, all the Go source files still reference the old git host, since imports in Go are URLs to a git host.

Reply on news.ycombinator.com