How to Host Your Own Private Git Repositories
eklitzke.org
eklitzke.org
Even if you are hosting on github/bitbucket/et. al. though, that repository is just one of many equals. You can push and pull from multiple peers as long as you have access set up appropriately.
I recommend the chapter on distributed workflows in Pro Git:
https://git-scm.com/book/en/v2/Distributed-Git-Distributed-W...
There's also an explanation of the different supported protocols here:
https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protoco...
Normally if you're hosting on one of those sites, you're using other features (wiki/issues), which may _not_ be decentralised
It has great instructions for installing directly from source and you don't really need to be familiar with ruby to install it. It requires some standard components (web server, database) which should exist on all servers and then you follow the instructions and presto! You have your own gitlab!! It also has great upgrade instructions so you can always be up-to-date.
I know gitlab can also be used through the cloud version (and it even has free private repos) however some organizations feel better if the source code of their projects stay inside the organization.
I have tried it before and it just works, took my less than 5 minutes to have a fully working gitlab instance on a $10/month droplet. It's not the cheapest option probably, but very low effort so it's quite worth it.
If you need something a little bit more complex, I would highly recommend gitolite for managing repositories & users. Configuration is done via some INI/TOML-like files in a git repo. User public keys are stored in the same repo.
Gitolite home page: http://gitolite.com/gitolite/
I run a multi-tenant gitolite setup for the University of Cambridge: https://git.csx.cam.ac.uk/
It has a pretty clever config management system where the configuration is actually committed to a git repo. It's great if you're managing configuration by hand, but was difficult to automate via chef.
In order to let users create their own repos, we enabled "wild repos". It's as simple as a `git clone` with the desired repo name, which is great except that users often make typos and end up accidentally creating typo'd repos. The only person who can delete repos is the user who created them or the server admin deleting directories. Perhaps there are better features I should have used?
Git repos can have an "update hook" that is executed during a push, after the new commits have been uploaded, but before the branch is updated to include the new commits. The hook can inspect the new commits, compare them to the old ones, and decide to reject the update. (It's a shell script, so every imaginable condition is possible.)
I did find http://gitolite.com/gitolite/list-non-core/#partial-copy-sel... which seems to almost do what I wanted but that'll quickly explode to many copies of copies for any non-trivial situation. I know git can't do what I'm asking but every now and then something comes along and I gives me some hope before reality sets in.
(You'll have to create separate UNIX shell accounts for all your developers sharing a common group with this setup, and this may or may not be a good idea.)
While the hardware below had some major failures (2 SSDs died during that period) the gitolite always survived. I like it :-)
It's the same as running postfix/dovecot for configuring your own mail server (which could run on a lightweight VM), or using a turnkey solution such as Zimbra (which will include spam/virus filters, LDAP, calendars and much more).
Depends on your organisation: how often do you create accounts? who can do the sysadmin work? etc. I'm happy to sit back and let managers handle account management, and to be able to put Gitlab/Zimbra on a job description if we need to hire someone.
Giving you feedback is a one-off; but the right density of UX designers in your teams and user interviews in your process will teach you much more.
I buy the need for the gitlabs of the world when multiple users show up, if only because managing credentials is a pain. But for single user use cases I wouldn't waste my time.
"Based on your choice, install one of supported databases or skip this step"
gitolite has worked well for small multiuser teams for me. No difficulty with managing credentials and no need for a heavy solution.
mkdir project.git; cd project.git
git init --bare
git clone user@example.com:project.gitOr I could work on problems that really matter and leave the sysadmin job to someone who gets paid to do it. ;)
More seriously, when I was in college, I spent a lot of time doing things like running my own mailserver, selfhosting various projects, etc. I learned a lot. But in the Real World, I don't want to be responsible for more than I have to be; off the shelf products are just better for me, most of the time.
The fact that Bitbucket and Github will pay a guy to run a git server for me is amazing (even if it is evidence of some sort of irrational enthusiasm on part of VC firms). Why would I not want to take advantage?
Edit: seriously, I wonder how many people who just automatically go to github have ever bothered to try the simple act of creating a git remote on a file server on their own network, or even just to a different host. It's really easy, and it really underscores how simple it is to have your repo distributed without any 3rd party infrastructure. Once you see that, you see that putting a copy on a shell account on your hosted VM is dirt simple and requires almost no administrative burden.
So what companies like GitHub and Microsoft provide is - yes - the fancy tools on top but also teams of professionals ensuring that your repositories are available quickly.
A private source repository is far less demanding: it's almost never upgraded, and for system administrators it's just another server to keep running and another file system to back up.
GitHub is distributing your Git repository across multiple servers in multiple racks in real time for reliability and availability and is the world's largest Git repository hosting provider. Some nice conference talks discuss this, like from Git Merge: https://www.youtube.com/watch?v=f7ecUqHxD7o and GitHub Universe: https://www.youtube.com/watch?v=DY0yNRNkYb0
Visual Studio Team Services is hosting your Git repository across Azure, and is hosting the world's largest Git repositories. https://arstechnica.co.uk/information-technology/2017/02/mic...
I have my own Git server as well, and I agree that its maintenance isn't very demanding. But I'm not putting production bits on it. My open source repositories go to GitHub and my private repositories go to VSTS - they're providing a level of service that I simply can't match by myself.
I have github and bitbucket accounts, but for little projects, I much prefer the simplicity of effectively just having dupes of my repos on other machines of mine. And it's really nice that the way I interact with them is precisely the way I'd interact with github or repositories at work, despite the fact that they're just directories sitting behind an ssh connection.
"Just another server to backup" is, indeed, not a big deal. It's strictly more work than not having another server to backup, however, and when it fails, it fails hard--in the sense of, "Hey, I just blew Saturday fixing my personal git server" or "Hey, I just lost data because I realized my backup cron was broken."
Running your own server is a bug, not a feature. The fact is, it's a very _minor_ bug because running your own is so damn easy. But it's still a bug. (The fact that you can run your own server, in the contrary, is a feature.)
Anyone with more than a personal pet-project will understand the value created by these services the moment they have to stand up the supporting items that make SDLC possible.
Using SSH on a local server works for some use cases (and I do use it) but it doesn't scale at all.
I personally set up a remote-access git repository on Windows, with Windows domain authentication (Apache+mod_ldap+mod_authnz_ldap), and it wasn't any more difficult than any other Apache installation; a more sensible platform would require a negligible effort.
I use GitHib client using BitBucket for repo hosting with LFS and it works great, no need to host anything.
Because you never know when one of these online services will suffer an outage, suffer a security breach that leasks your private repos or email & credentials, or even lose your data entirely. The fact that the service is free doesn't mean that it doesn't come without potential issues.
I don't know... at some point you sort of just have to trust someone... be it the hosting provider, or the service provider. And I'm old... but I've had to untangle issues with CVS / VSS / SVN / etc... over the years I don't want anything to do with that crap -- if I can punt it to someone else to manage I'm OK paying some tiny subscription fee.
I think there's been one day in the last ~10 years when I couldn't use GitHub. To me that seems worth the $20 a month, or whatever they charge now.
Fixed that for you.
Not everything needs to be in the cloud, and using any online service run by someone else brings some degree of reliability, security, privacy and longevity risk. Some of us just prefer to avoid those risks, and usually will unless there's some compelling benefit that outweighs them.
Turns out LFS is designed to authenticate over https and doesn't work at all with ssh credentials out of the box because f* you.
I'm still hoping for a native git feature for large files, so the git-lfs crap can die in a fire.
Compare this with the Mercurial LargeFiles extension, which needs nothing more than a line in the .hgrc on each end (client/server) to enable it.
AWS CodeCommit:
5 active users per month
50 GB-month of storage per month
10,000 Git requests per month
Does not expire at the end of your 12 month AWS Free Tier term.
https://gitbucket.github.io/gitbucket-news/gitbucket/2017/03...
This is only an issue if you're sharing the box and/or remote repositories with other people. For shared remote repositories I've been using the following setup:
1. Create a bare, shared repository at `/var/git/foo`. Configure unix group permissions and the directory setuid bit on it.
2. Give alice access via a `/home/alice/foo -> /var/git/foo` symlink.
3. Set alice's shell to a patched version of the git shell I call `git-home-shell` that sanitizes the repository path argument and makes it relative to her home dir.
Is there a better way these days?
You could create a group for each repository, and add and remove members as necessary.
But, I'd prefer it if git-shell didn't let users probe and read git repositories at any absolute path on the remote end. That's not great behavior for a restricted shell.
That's a lot more work than a restricted shell which just...restricts.
Personally I'd just designate a path for shared repos (e.g. /srv/vcs/<project>{.git,.hg} etc), give people write access using ACLs and group membership. If they create repos in their home directories, thats their business.
If I may make a suggestion, I'd recommend the Super Dimensional Fortress Public Access UNIX System (https://sdf.org/).
They're NetBSD-based if I remember correctly, and for a low fee (36$/lifetime ARPA membership + 9$/quarter) you can host most of the things you would like to host.
And you don't have to do system maintenance.
edit: thanks anyway for your version!
Besides, if you worry that state actors are interested in your source code, I do not see how Amazon being law-abiding would be of any help here..
To simplify the initial setup, I created a handy shell script, reposetup [1]. It makes to create repositories, push to them and remind me their urls.
There are some tools that restrict access with varying levels of granularity, but if you just want to restrict access on a per-repo-per-sshkey basis, one of my projects is a simple shell script that does just that:
https://github.com/cbdevnet/fugit
It originally came to be because I've found gitolite too big to maintain for simply sharing some repositories with a few other people. It has since served me well and is used in some business applications, too.
(I know it doesn't look complicated, but if there's a decent "standard" already out there...)
Also, for backup, rather then tar up the ".git" directory, I use "git bundle <backupfilename> --all" which creates a flat file with all branches included. This file can then be uploaded to GCS or S3.
https://stackoverflow.com/questions/1960799/using-git-and-dr...
Is a git extension to allow a Dropbox to be used as your remote. Works from the CLI. Been using it for two years.
A little advice: it's probably best to post a disclaimer that you are the founder of RhodeCode whenever you post about it on HN. OTOH, you get credit for including that info in your HN profile.