Hacker News Reader

Comment by Xylakant | Hacker News Reader

> Anyone else remember the days when having your SVN server go down meant that you more or less had to wait until the server came back up before you could get back to doing things? (if you wanted a nice atomic commit).

That's now known as "I can't deploy because github is down". Note that the author proposes "local commits plus a centralized storage".

> The author mentions that you have to retain the whole history of a project. For one thing, storage is cheap.

Not in portable computers. I can get a TB, but that's it. I have build artifacts that clock in at about 300 - 500 MB and I'd version control them if possible. I can't, because that would fill my disk within a couple of month, so I have to push them to a server and somehow link them.

> Anyone else ever needed their own fork of something due to differing goals from the parent project, but needed to fold in upstream changes relatively often? That's by nature the sort of problem that a DVCS solves easily.

That's a strawman. The article does not argue that there's no use-case that's solved by git or and DVCS. It's just that not every use-case is solved by git. I'd even go out and argue that most use-cases are solved just as good by a solid centralized system with less complexity involved.

> Just because we haven't had tools in this space with wide adoption that are user friendly doesn't mean that we won't eventually.

I see a developer wedge his git repo with a pull + rebase about once a month. And then somebody needs to walk over and explain. DVCS fundamentally introduce complexity that is not always needed. And I doubt that the fundamental complexity can be abstracted away.

View on HN

> That's now known as "I can't deploy because github is down". Note that the author proposes "local commits plus a centralized storage".

And in that cause you can scp your repo somewhere and change your origin and move on.

Also, do people really rely on 3rd party services for their deployed code? I've used github as a repo and colab tool at companies, but we still deployed from repos owned by the company.

> I'd even go out and argue that most use-cases are solved just as good by a solid centralized system with less complexity involved.

I'd go out an argue that because the added complexity of git is far outweighed by the benefits of a local repo.

    Also, do people really rely on 3rd party services for their deployed code?

Yes, this is very common.

Also the fact that each developer has a full copy of the repo on their laptop lets me sleep at night wrt github exploding.

Or your company exploding or pretty much anything exploding. When every dev has a full copy of the repo, you can restore all the work as long as a single dev machine is alive.

Not to mention that in case of an office Internet outage, you can just mesh together those repos and continue working. Knowing that gives me some peace of mind.

It also means that you get to loose control over all data in the repo including all versions and history when a single person looses the laptop, even if it's a person that only required read access to a tiny part of the data.

It's a valid tradeoff to make, but still something I'd keep in mind.

I'm going to go ahead and say that that's a red herring.

A successful business is built upon so much more than whatever code is in your repo.

Any competitors are probably more likely to look at the code and ignore it over their own home-grown solution than to adopt it wholesale...and if they can beat you with your own code, they would've beaten you without it, because they probably run a much better business game.

Trade secrets are like the stupidest reason ever to hobble your developers.

I think you're wrong on that account, there's tons of pain that can be caused by such a loss. But I was thinking more about the SSL certificates and SSH keys and hardcoded passwords and other secrets that inevitably seem to end up in repositories. (Rails cookie secrets anyone?) Some of that data gets deleted but not purged correctly and oops, there's that one server that still accepts last years key, didn't we delete that from the repo? I've seen some repos that could easily create damage in the 6-7 figure range if they ended up in the wrong hands. Enough to sink a small company in any case.

All of that is configuration, not code, any should never be in any repo external to one's company. Nor, frankly, should it be available to the developers. That's all production-specific, and should be restricted to deployment.

If your developers are deploying into production directly from their laptops, you have a problem.

So, admins never use version control for puppet manifests/chef cookbooks/terraform/saltstack/whatnot and all the config data that's put on the servers? And it would not be of value if we could version the configuration along in the same repository as the code to which it belongs? So it's easier to track? Now, currently we can't, because DVCS don't allow us to grant different privileges to different people/groups, but I could very well imagine that developers get access to a repo where they can change the configuration to the dev/test environments and the admins match that in the same repo for prod and other envs with higher level needs. I could also imagine building the final deployment from a single repo instead of having to merge two repos because I need to separate them due to permissions.

I could also imagine designers placing their assets in the repository without needing to deal with git or even seeing the source code that lives side-by-side. That totally used to work with SVN I'd totally like it if the repository contained the PSD sources for all the assets/mockups that are relevant for a given version of the source, but alas, you can't check out the source without getting the PSDs, even if you're not interested in them. Heck, SVN even allowed exposing the repo as webdav which you could mount as a networked folder, so people could access the last revision just as if it were a SMB drive.

git is a huge step forward in some regards, but we also lost quite a couple of good things along the way.

> If your developers are deploying into production directly from their laptops, you have a problem.

I never said they do.

So, the performance of binary assets in SVN is exactly why we used to--when doing game development--check all the source files out into their own repository, and occasionally sync over.

As for the configuration being in the same repo as the code--again, why? That tells me that you aren't packaging releases, or if you are, your documentation is less useful as a reference than your code.

These are antipatterns, again, and "fixing" your VCS isn't going to help in the long run.

> So, the performance of binary assets in SVN is exactly why we used to--when doing game development--check all the source files out into their own repository, and occasionally sync over.

The assets for game development likely are bigger than the assets in web development but we never had major issues with binaries. Things also improved much in later SVN versions (in the beginning the client was way to stupid and tried to diff binaries which obviously could not work)

> As for the configuration being in the same repo as the code--again, why?

Why not? I don't have that right now, but I'd like to have it. Because I could compare the state of the code and the state of the configuration with the state of the code easily at any time at any commit. I could use bisect to figure out at which point things started failing. Helps when you're trying to figure out when config and code started drifting apart. I can use google's repo tool or submodules or stitch together the config repo and the code repo by matching up dates or tagged versions, but that's all a hack. It's maybe the best hack currently available, but I still think there's room for improvement.

> That tells me that you aren't packaging releases, or if you are, your documentation is less useful as a reference than your code.

I can't imagine how you reach that conclusion. You could not be further from reality. Just because I'd consider it nice if I could have a single version identifier in a single repo that matches up both, config and code I'm not rolling that in a package? And how does that relate to my documentation?

> So, admins never use version control for puppet manifests/chef cookbooks/terraform/saltstack/whatnot and all the config data that's put on the servers?

Of course they do. But those repos should live on an internal repo server, and on the production systems themselves.

Copying production authentication credentials to a non-production machine should be a firing offence.

> And it would not be of value if we could version the configuration along in the same repository as the code to which it belongs? So it's easier to track?

Hell no. Code without configuration is like a gun without bullets: it's an interesting piece of work, but that's it (I'm not a huge believer in intellectual property, you can tell). But configuration is the keys to the kingdom. If a cracker has your app's code, he might be able to figure out some flaws in your protocols or your implementation, but if a cracker has your database passwords, your hostnames and IP addresses, your firewall configuration, your bastion hosts—then you're dead.

Enlighten me, I'm seriously interested in knowing, because I'm missing the point:

Currently there's code in one repo and configuration in another. Both repos are on the same internal repo server. Both repos are accessible to multiple persons - some can access the code only (developers) and some can access the configuration repo only and some can access both repos (admin). A build process tags both repositories and builds an artifact that gets deployed.

How is that situation superior in terms of security over:

Code and configuration live side by side in the same repository that supports access controls. The repository is hosted on an internal repo server. The parts that are code are accessible to developers only and admins can access both parts. A build process tags the repository and builds an artifact that gets deployed.

The only point I could see is that with two repositories it's harder to mess up the authentication, but I doubt that's true. In both scenarios we have people with access to the configuration and those people will in a lot of organization have a copy on their laptop that they carry around. That's how most people use version control. In both setups it would be possible to only ever handle the sensitive data on a remote system, but that's a property of the workflow and not a property of the VCS used. I seriously don't see the issue with a shared repo - if it supports access controls.

Well, you do that anyway when you allow people to check out a local copy of the code. Just as in most VCSes, you can set up a git server to only allow checkout of specific branches.

>Well, you do that anyway when you allow people to check out a local copy of the code.

You grant that permission to the person legitimately checking out code, but not to the person finding or stealing a laptop with a clone of a repository. The latter is a side-effect of how a DVCS works. In SVN you don't even need to expose the full history, you can grant access to the last revision only.

> Just as in most VCSes, you can set up a git server to only allow checkout of specific branches

In SVN for example you can restrict people to single directories (or even files - I don't remember exactly). That at least is impossible in git. I can prevent pushes using hooks but not reads.

> You grant that permission to the person legitimately checking out code, but not to the person finding or stealing a laptop with a clone of a repository. The latter is a side-effect of how a DVCS works.

I'm not sure what you're getting at. What difference is there (not that you would allow checkouts on unencrypted laptops anyway)?

> In SVN you don't even need to expose the full history, you can grant access to the last revision only.

> In SVN for example you can restrict people to single directories (or even files - I don't remember exactly). That at least is impossible in git. I can prevent pushes using hooks but not reads.

These restrictions may be useful in some cases, but I would wager that they are far more seldom than some of the advantages of git (like being able to work offline).

> I'm not sure what you're getting at. What difference is there

A checkout from SVN/CVS only contains the last version. Files that were deleted in an earlier version are only on the server. A clone of a DVCS contains all versions and all files that ever were in the repo (unless you use BFG or git-filter-branch, but people tend to forget that). So a clone can contain secrets that people are not aware of, such as accidentally committed and deleted files. An interested party could find stuff that you're not aware off by looking at HEAD.

> (not that you would allow checkouts on unencrypted laptops anyway)?

That's not my call to make, but I agree on that regard. Reality sadly different from what we both wish.

> An interested party could find stuff that you're not aware off by looking at HEAD.

Well, that goes without saying. But I don't think that security argument is a very poor one compared to the huge benefit of having the history locally to inspect.

We've had instances where secrets were committed to local repositories by accident. It never got past review and into the master branch. If it had, we would probably had taken the effort to rewrite that commit out of the history.

> Well, that goes without saying. But I don't think that security argument is a very poor one compared to the huge benefit of having the history locally to inspect.

If you go further upthread you'll find that I said "a valid tradeoff, but one I'd keep in mind"

> We've had instances where secrets were committed to local repositories by accident.

That's laudable, but countless examples show that not everyone is that diligent. I'd love if I could lock down some parts of some repos so that they're only accessible by people that I have an elevated level of trust in. (and where I can enforce a certain security level on the laptop).

> That's laudable, but countless examples show that not everyone is that diligent.

Sure, then again I would guess that the ones who are not that diligent are not likely to apply those access restrictions that you mention (although the "one revision" advantage is something they would get "for free" with SVN).

In any given larger organization there are people that have exert control over only parts of the whole. I could possibly argue to tighten down security on parts of a repository for some people within boundaries (like declare some folders as unreadable to some folks that don't need access to those) but I can't deny them all access since they need some of the content stored in the repo. With git that's currently all or nothing which exposes a flank that I'd prefer closed. In this particular case it's not a terrible issue, but for other folks with other data that can quite well be, so the tradeoffs may end up being in favor of SVN. I can imagine that that's one of the reasons I still see SVN deployed in corporate installations.

Or, you could use Mercurial and sleep even better knowing that not only does every dev have a full copy, but does NOT have thousands of orphaned files of each revision or potential revision like Git does.

Cloning doesn't require the full repo. Some of your devs may only have half of the repo. Look at the --depth option for git-clone.

Hell, for a ton of companies (including mine) everything is on 3rd party services/servers. We don't have any servers we can actually call our own.

Which is fine, but the very obvious and up-front trade-off of using servers that aren't yours is that they aren't yours, and everything that implies. There are plenty of benefits as well, but that's why it's a trade-off.

> And in that cause you can scp your repo somewhere and change your origin and move on.

If you happen to have - god forbid - a php app that pulls dependencies via composer you have a hard dependency on github since composer pulls practically all code from GH. I don't consider that a good idea, but that's how it currently is. [No, I don't do php and only sometimes have to clean up the resulting fallout]

If particular frameworks want to tie themselves to a single server/host, that's their problem, and is specifically not a problem with any of Github, git or DVCSes generally.

You really cannot stop all people from being idiots all of the time.

That's not a hard dependency, it's a sensible default that you should be overriding if you're running composer install as part of your deployment process instead of including composer's vendor directory in your repo. Documentation: https://getcomposer.org/doc/05-repositories.md

Practically all packages use git sources. Yes, you can vendor them - congratulations, you just kicked the can down the road and moved the issue to the build step. The issue exists for other languages as well - maven/mavencentral, ruby/rubygems.org - but only composer depends on github very much. I don't like that but it's not like the other solutions are much better.

Many companies mirror maven, so having mavencentral go down is not a big issue. Does composer make it possible to deploy a mirror of its dependencies?

You can use your own repository, so at least in theory that should be possible. I haven't done that in practice, so I can't comment any further.

Yes, certainly: https://getcomposer.org/doc/articles/handling-private-packag...

I would bet its dependency is git not github.

I'd go out an argue that because the added complexity of git is far outweighed by the benefits of a local repo.

But for precisely what use cases or in what situations? If one is a novice and just want to have source control and versioning, centralized systems are going to have the better cost/benefit.

> If one is a novice and just want to have source control and versioning, centralized systems are going to have the better cost/benefit.

This seems completely backwards to me. When I was in college, I thought it was a giant pain to source control my homework, because the options I was aware of (since all I knew about was SVN) were 1) public hosting on sourceforge (which was itself a pain to use), which professors were not keen on, 2) find a private host somewhere (not sure I ever did), 3) run my own server somewhere, 4) configure and run a local server. I went with (3) and (4) but it was never an easy or novice-friendly solution, and mostly I just didn't version control things. The local version control model of DVCS would have been much easier.

It's only when working with other people that any extra complexity even rears its head at all, which doesn't come up if you're a novice who just wants to have source control and versioning.

a purely local svn repository never needed a server.

svnadmin </path/to/repo/on/local/disk> followed by svn checkout <file:///path/to/repo/on/local/disk> is all it ever needed.

Same for git, though.

> Not in portable computers. I can get a TB, but that's it. I have build artifacts that clock in at about 300 - 500 MB and I'd version control them if possible. I can't, because that would fill my disk within a couple of month, so I have to push them to a server and somehow link them.

git-annex is a pretty good solution for this: https://git-annex.branchable.com/

> That's a strawman. The article does not argue that there's no use-case that's solved by git or and DVCS. It's just that not every use-case is solved by git. I'd even go out and argue that most use-cases are solved just as good by a solid centralized system with less complexity involved.

I'm not convinced that it is a strawman– I'm not an uber-developer, but I've had to do it reasonably often. A common case is that the originator loses interest, and you still need to do maintenance on it. I'd rather my go-to choice of a version control system support that notion rather than learn a different tool to deal with this case.

> I see a developer wedge his git repo with a pull + rebase about once a month. And then somebody needs to walk over and explain. DVCS fundamentally introduce complexity that is not always needed. And I doubt that the fundamental complexity can be abstracted away.

Sure, there is an additional level of abstraction, but to argue that it's insurmountable seems rather pessimistic. Most technologies that we take for granted now required multiple decades to get to a consumer-friendly state. This is perhaps a philosophical difference that only time will answer.

I'm not a fan of solutions like git-annex, every time I've seen people try to do this type of stuff, two years down the line they end up having built a buggy knockoff of maven/gradle. Why not do things cleanly from the start? The entry barrier is really not that high, and the benefits are huge.

Then again maybe I'm a bit psycho-rigid on build management :)

You're absolute right from my experience. I"ve seen many shops using svn:externals to tie build dependencies together. And now that git is starting to see wider adoption within industry (not just software companies) the same is attempted with git submodules and git annex. It's horrible because this is not the use case these features were designed for, but people try anyway and eventually run into road blocks.

This tends to happen especially in shops that migrate off process-heavy monsters like clearcase and MKS. These tools do a lot more than just version control, and no open source version control system by itself matches what users of these systems expect. You have to throw at least an issue tracker and a good build tool into the mix as well.

Totally agree, big organizations need something integrated. With GitLab we include the issue tracker and build tool you mentioned as well as having git-annex for large binaries.

Have you checked out ipfs?

> The entry barrier is really not that high, and the benefits are huge.

Because of "we'll just add a submodule and we don't need a build system and it's only a single one and we'll replace that with a proper build tooling as soon as we have time". And then it's only a second one and suddenly submodules pop up like mushrooms after a light summer rain. It requires planning and forethought, just like a lot of things that should be an obvious huge benefit, but planning and forethought are in short supply in this industry.

What are you suggesting the right way from the start is? Just using maven/gradle?

A million times yes, if only for the dependency management side. Can you remember the dark ages of having to download libraries from the internet and check them into your source? Arghhhhh

git-annex is a tool for versioning files with git that you don't want to actually store IN git. It is not a build automation system.

That's now known as "I can't deploy because github is down".

git != github. I use git every day at work. I don't use github for that repo. If I did, I wouldn't make reliance on it part of my deployment strategy. If I did do so, I hope I would realize that any problems relying on github resulted in were ones I created with that choice. Hopefully there would be some useful benefits from that choice as well.

Why in the world would you check in build artifacts into source control, especially those that clock in at 300-500 MB?

Because you are accustomed to centralized version control systems that do not punish you for doing something that dumb, and refuse to learn new tools when you are forced to use them.

Anyone checking in 500MB artifacts into git is almost certainly refusing to use git correctly.

It is like somebody who grows up using hand tools, gets handed a power drill, then tries to use it like a pry-bar. You should not use a screwdriver as a pry-bar, that is an abuse of the tool. Nevertheless, many people abuse screwdrivers as pry-bars because conventional hand screwdrivers tolerate this practice.

Thanks for assuming I'm an idiot without knowing the constraint I'm trying to fulfill ;)

> "almost certainly"

You might have a legitimate reason for putting 500MB artifacts into a git repo, but I reeeaaally doubt it.

It is a poor craftsman who blames tools that he is intent on misusing.

You're putting words in my mouth that I did never say. I'm not blaming git for anything nor do I consider git a bad tool. I'm using git for practically all my versioned data. I just don't consider it the tool that needs to solve all cases where I - legitimately or not - want to version data. A good craftsman should have more tools at his disposal than just a blunt hammer.

So you can doubt my use case but I consider it severely impolite to pretend that you know better. Actually you're quite nicely illustrating one of the points in the article.

And you're still not even trying to offer a real explanation, just engaging in a flamewar.

It's all discussed further down in the tread, and I gave as much info as I could. I'm not at liberty to discuss details in public - nor do I need to.

Further down on the page somebody else also mentions a legitimate use case to version-control large binaries (needed for comparison). Another use-case I've seen is version control rendered video output and keep the comments and metadata attached to the versions. Works just fine with SVN, fails hard in git. Yet another use case for a system that handles binaries better is what the rubygems folks do - they vendor all gems that a particular version of rubygems.org depends on so they can bootstrap without rubygems.org being available. They built a custom solution using multiple git repos which works for their use-case (it's been discussed on rubygems). Arguably, having a system where versioning large amounts of binary data works better than in git would have prevented that issue.

So there are use-cases that are ill-suited for what git can and cannot do - and just because I say I have one I get to be called a bad craftsman by someone who doesn't know the least bit about what I'm trying to do?

And now you're saying that I engage in a flamewar when I point out that I consider that an insult? Please note that I have not insulted Crito.

The real explanation is that someone would want to version binary data. Like say, images. The parent poster claimed that anyone who would want to version binary data is an idiot.

After a few days please review that comment of yours. Could anybody in an internet conversation really know all details of your constraints?

Maybe, maybe(!) there is an argument were you would need to version huge binaries which you could generate out of the same already versioned sources, but even if there is, there are methods to backup huge blobs, and git is simply not one of them.

> Could anybody in an internet conversation really know all details of your constraints?

No - nor do they need to. But everybody on the internet could just assume that _I_ know them and refrain from saying that "I'm doing something dumb" and "refuse to learn new tools". So I'd also ask you to review the comment of yours - you're doing the same thing - you imply that "Maybe, maybe(!)" there might be a use-case, so as a matter of fact you doubt I did review and choose my tools.

I reviewed my comment, and I'm at peace with it.

In that case, why did the parent poster assume anyone trying to check in a large binary file is "an idiot"?

The "idiot" part of course is wrong. But if you understand a technology and then use it for something it is explicitly bad at, than it's objectively a bad idea. The chance is very low that you would not be able to solve a binary sharing/backup problem with another tool that's made for these tasks.

The original article addresses this.

"These are large, opaque files that, while not code, are nevertheless an integral part of your program, and need to be versioned alongside the code if you want to have a meaningful representation of what it takes to build your program at any given point."

In this case, git isn't the right tool for the job. Large gaming companies, IIRC, use Perforce for this reason.

> I see a developer wedge his git repo with a pull + rebase about once a month. And then somebody needs to walk over and explain. DVCS fundamentally introduce complexity that is not always needed.

Rebase and DVCS are orthogonal concepts. Rebase is a consequence of offline commits, not of decentralization. And I greatly prefer dealing with a `git rebase` issue than the equivalent in Subversion, which is to have your uncommitted local changes get irrevocably modified by the `svn update`, with conflict markers placed in your code and no way to abort the whole process.

But... but... but... git bad!

It's a prevalent meme, isn't it? I'm surprised at how many people never even realized how fundamentally dangerous it was to issue an `svn update` with local changes. And perhaps less seriously but still problematic, how `svn commit` can put the repository into a state that never existed on any developer's machine (because it effectively does a rebase on the server), which is not always safe to do even when there's no merge conflicts.

> how `svn commit` can put the repository into a state that never existed on any developer's machine

git add -p creates the same issue for git (a commit that never existed on disk) and git rebase with squash and/or reodering of commits does as well. That's usually considered best practice.

The problem I often observe with git (pull -)rebase is that people have a hard time wrapping their mind around the fact that all commits change and when they change and in what state the checkout is are when they get a merge conflict (remote HEAD plus whatever was already applied). The conflict is often unavoidable and you can argue that rebase is making it easier to resolve the conflict, but it's harder to reason about than "ok, update does a merge and I get whatever is on the server plus my local changes".

The difference between `svn commit` and the various forms of `git add` or rebasing is that the developer can test the commits (or visually inspect them) before pushing. In fact, the argument that `git add -p` produces a commit that didn't exist on disk is exactly the same as the argument that developers can commit code that they never actually built & tested. Which is to say, the important point isn't that the commit tree existed in isolation on the developer's disk, but rather that the developer did any testing necessary to verify that the commit is good. And the reason why `svn commit` is bad in this regard is because it creates the never-before-seen result on the server.

> The conflict is often unavoidable

If the conflict is unavoidable, then you're going to get a conflict regardless of how you go about it (rebase vs merge vs `svn update`). The difference is with `svn update` you can't abort the whole process and start over if you need to, because you've already permanently lost the previous state of your work tree. Whereas with `git rebase` and `git merge`, you can abort and you'll get the exact state you had prior to the command.

Why are you putting build artifacts into source control?

Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result.

Why are you asking?

People get pretty religious about what should (or not) be checked in to version control.

I can tell from the downvotes :)

It's designed with a particular use-case in mind. When people complain that square pegs don't fit into round holes, it makes more sense for them to step back and evaluate what they are trying to accomplish, and the tools they are using the accomplish it.

I think the question asked was closer to "why aren't you using an artifact repository?"

Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds.

Valid question: Because the tooling does not think in terms of artifacts that are versioned in repositories, but rather in terms of "files" that are in a given location. I'm using a shared location, but every build requires modifying another file to point to the now current version. It's all solvable, but the easiest solution would be to just version the result.

I could fix the tooling, but alas, I have other yaks to shave. It's an imperfect world.

> but every build requires modifying another file to point to the now current version

If you could version the file in git, you would have to check in the new version of the file, so it's not like you're adding a step to have to update (e.g.) a symlink.

But I have a lot of files lying in a shared location that are named build-<datetime> and I can accidentally break revisions in the repo by moving/renaming/deleting any of these. That may be a feature to some people, but that's something I consider a weak spot. It's brittle and prone to breakage and I dislike brittle.

> Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds.

Here's where I start to have problems with contemporary development culture. You mentioned using Nexus, Gradle, Maven and Jenkins where the guy just want to keep some binaries along with the source code they're generated from.

We're complicating things beyond reason nowadays.

Agreed.

That said--and this is without knowing the exact build and tooling environment, so I may well be giving advice inappropriate to the situation at hand!--the second part of that "keep some binaries along with the source files they're generated from" is kind of an antipattern.

If it takes too long to generate them from source, each time erry time, they need to fix that issue--least of all because slow builds mean slow testing, and slow testing means no testing.

That's why I spent two days earlier this year moving a 30-minute build down to a 2.5 minute build.

Yes, your advice is sound for the general case but inapplicable in this specific incarnation of the problem.

Out of curiosity, what is the gist of your setup? Why is this incarnation such a departure from the general case?

The output of the build is basically a tool in itself. So most people don't need the build process, just the resulting tool. The input changes on pretty much a monthly basis and is not easily versioned. I could set up all dev machines to support the build and everybody could build it themselves from sources, but that would require me to

  * install all the required instruments for the ritual rain dance
  * teach the whole team how to do the ritual rain dance
  * support the people that break an arm or a leg doing the ritual rain dance

So I prefer perform the dance on my machine, collect the tool and point the main dev environment to the right location. It's all scripted, so I kick off the job and grab a coffee. However, I want to keep old instances around so I can track when bugs crept in, so I can't just go overwrite the result, so I need to adjust the pointer every time. If I could just check the tool in with the regular dev setup that would be much easier, but - since we're using git - that would blow up quite quickly and overwhelm my disk space. (And folks would kill me for filling up their disks as well - rightfully so). That's something SVN or another centralized VCS would handle much more gracefully. In a gist, I have a use-case for fairly stupid versioned file storage with a push/pull api. No complicated merging, no branching, nothing. git-annex could do, but is overkill.

There are some better solutions to what we're currently doing, but there's so many yaks to shave and so few razors.

> That's something SVN or another centralized VCS would handle much more gracefully.

Subversion handles this gracefully because you don't download all of the repository's history to your machine. It's a trade-off that you're talking about here. Most people are ok with losing the ability to check in 500MB files in order to gain the decentralization of having a full copy of the repo (and not needing to query the server just to view history).

Thanks for such a good explanation. When I first saw artifacts of things generated by code in our repo, I had a big WTF moment, but it made a lot of sense once someone pointed out that it was Rather Handy for catching bugs in the code that does the generation.

What if, instead of binary build artifacts, they were images?

He also mentioned using a simple shared drive. It would have been difficult to know which of these solutions was the right level of complexity and capability without knowing more context (a point which he seemed to make clear).

Shared drives are mutable which is a big problem with build output.

No, I didnt. I mentioned a shared location. It's not an networked drive, I'm not insane.

Fair point. Comment retracted. My apologies.

To bring it back to the OP, this argument is in fact represented in the OP.

OP is arguing that these (having to use Nexus, Gradle, Maven, and Jenkins just to keep some binaries along with the source code they're generated from) are workarounds to limitations in git that ideally would not be there (and don't neccesarily have to be there, and aren't there in all VCS's), and the OP mentions that instead git fans want to claim "No, that's just the way git SHOULD work, you SHOULD need to go use an 'artifact repo' in addition to git to keep a few binaries with your source code" instead.

I tend to agree with the OP.

> We're complicating things beyond reason nowadays.

There isn't always a way to dumb things down to the level that people would like. It would be so much easier to get to work if I could fly in a straight line between work and home, but I don't complain that the world doesn't accommodate me.

>> Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result.

Put your build tools in a repo. It should be easy to set up a new developer with a complete build system by getting it from the a repo. Now use make or anything that can check dependencies so those things are not regenerated unless they need to be. Always check in a freshly generated file along with its source.

Setting up a build system can be a bitch. Not every developer needs to be able to build every obscure module of all your companies tooling. A .net developer shouldn't have to rebuild c++ boost just because he relies on a small native dll.

In some cases the tooling of one developer can even conflict with the one of another, one requires python 2.7 to be in your PATH while the other requires 3.3, etc.

I fully agree that it should be easy to set up but the reality is different, especially when you deal with legacy, rely on third party libraries or build open source projects from source.

What is annoying is when build artifacts are checked into source control systems along side of the source. There is very little way to exclude them or only check out the binaries, you can only have both. This happens all too often and I don't care if you want to keep using subversion you really should stop doing this.

/src/foo.c /src/foo.o

Not the OP, but in our product we have very large binaries that are the output of the product itself. Each time a change is made to the program, the output needs to be compared to the existing output first automatically and if the output isn't byte-by-byte identical then manually (visually).

So essentially, without large binaries in source control we'd lose testing.

I think you argue in the same direction I often argue. Git is not for everybody. If you don't want to spend the time to learn git you probably are better of using something else.

Additionally I would like to add something: There are more cases were you need than you think, especially if you haven't learned it. E.g., you start a little prototype to present to your boss. It succeeds, boom, ten years later you have 100 man team working on the same prototype you started 10 years ago. Suddenly you really need forks, branches, some people spend their whole day merging stuff, etc. But hey, you can't switch from SVN because you have chosen it in the beginning and now everybody is using it, all kinds of scripts, tools, and optimized workflows require that you continue to use it.

Some people might think that spending two weeks in the beginning to learn to use a power tool is much better than starting earlier and thereby getting years of pain later on.

I agree: whenever you find yourself saying "I don't have time to learn this advanced tool that will make my life better, I have to ship now!", you should really take a moment and carefully verify that this is a good choice. (Sometimes it is a good choice though.)

In the world of science, I have colleagues who have said this for ten years wrt. LaTeX vs. Word, or even just learning EndNote vs. typing up reference lists by hand. I cringe every time I watch them spend an order of magnitude more time on pure overhead, and always getting stuff wrong, for every single paper they put out.

That's a really good reason: When you are unable to learn it correctly (maybe simply because it's too complicated). With the simple tool you are still not able to do all the cool stuff, but at least you can get something simple done fast.

> That's now known as "I can't deploy because github is down".

I had that happen a few months ago. Had to push the repo to bitbucket and change where a config file pointed to. It took almost as long as doing a merge in SVN.

> I have build artifacts that clock in at about 300 - 500 MB and I'd version control them if possible. I can't, because that would fill my disk within a couple of month, so I have to push them to a server and somehow link them.

Those are very unlike source code. You're not going to want or be able to diff them, or blame them, or view their history. A different tool is appropriate.

> I see a developer wedge his git repo with a pull + rebase about once a month. And then somebody needs to walk over and explain.

Yeah. I see that at about the same rate I saw developers lose data back in the svn days. Git can "wedge itself" but never in a way that induces data loss, IME, in stark contrast to its predecessors.

> view their history

You can view the history of binary blobs in a git repo.

The main reason for the abandonment of centralized version control was as much about Git and Mercurial having the advantages of DVCSes as it was about SVN starting to show its age.

I found SVN literally painful to use, perhaps because it was based around CVS's standards, and CVS was originally a series of shell scripts written in 1990.

> That's now known as "I can't deploy because github is down".

No. No it's not. "I can't deploy" is different than "I can't work," unless you "move fast and break things" by deploying as soon as you push your code. Even then git allow you to continue to work, even if you haven't deployed your code.

"I see a developer wedge his git repo with a pull + rebase about once a month. And then somebody needs to walk over and explain."

Accidents happen, but the pattern of making people make highly predictable mistakes and then calling in the experts is the result of not investing on basic training (and possibly not hiring smart enough people, but it's another matter).

By the way, that "wedged" repository in all likelihood has an uncorrupted working copy containing all changes and can be repaired offline: Git is technically so superior to Subversion and TFS that it isn't even fun.

How do you get local commits plus a centralized storage without basically building a DVCS?

Local commits do not require the full history, only a known base. So you could in essence store the commits relative To the last pull from the server and push those later. No need for full history.

I have to agree here. You only need a startpoint. In essence a commit is a diff on that startpoint. So "cloning" a single commit should be enough to create a new one and push it.

git has 'shallow' clones, where it checks out the tree as it was at a particular commit, and has a dummy commit which replaces the history of the repo up until that particular commit.

You could indeed do that. But now you have two different kinds of commit that can conflict in surprising ways. I very much doubt you could manage this in a way that's simpler than what git does (remote branches and local branches are branches, you can merge one into the other using the normal tooling you use for merging branches).

I'm fairly sure that at least on the ui-side you could improve over git, but the goal is not necessarily lower complexity. A semi-distributed system could have other advantages, for example fine-grained access control (read permissions on folder or file-level), support for partial checkouts, better support for large binaries (only the last revision on the local repo, older revisions on the server) etc.

git currently can't do that because of the way it's designed and built.

A DVCS without complete history is still a DVCS. In fact, what makes a VCS distributed is much more local commits than local history. At least on my opinion.

But well, anyway we decide to call that beast, a VCS with distributed commits and centrilized (fetchable) history, with a simplified evolution graph (created by the point of view of a central server) would bring 99.99% of the benefit of a centralized VCS, 99.99% of the benefit of a DVCS, and almost no problems from any of them.

git practically already has this with the --depth option to git-clone. The only thing that it's missing is the ability to enforce storing the last single commit.

I'm sure that you could probably build the system that you are describing on top of git, but you would still run into problems with large files and local commits, even if it was just "commits since last pull/push."

Yes, git almost has the shallow repository done. It's even in the article, and that "almost" means it's almost trouble free, but will catch everybody off-guard once in a while.

The other part about the simplified evolution graph is completely missing. I don't think one can solve that without completely rewriting the protocol.

Reply on news.ycombinator.com