GitHub is down
githubstatus.com
githubstatus.com
Whatever skill level I have in this area, I cannot nail down if it's from multiple "baptism by fire" situations, learned naturally via university courses in the scientific field, strategy video games (half joking), or other sources I cannot think of.
I've listened to some very impressive people handle serious crisis situations and I'm both in awe and curious how they achieved that level of deductive reasoning.
git remote set-url --add --push origin git@github.com:Foo/bar.git
git remote set-url --add --push origin git@gitlab.com:Foo/bar.git
see: https://git-scm.com/docs/git-remote#Documentation/git-remote...
[0]: https://cets.seas.upenn.edu/answers/git-repository.html
[1]: https://blog.osdev.org/git/2014/02/13/using-git-on-a-synolog...
[2]: https://git-scm.com/book/en/v1/Git-on-the-Server-The-Protoco...
I've used this on NFS drives, but also SMB shares from windows, and just about anything that can be mounted to a folder. Having an external hard disk drive or usb stick also works.
And lastly, git also comes with a daemon mode which makes it easy to temporarily host a server for a repo. Just connect multiple laptops trough Wi-Fi, and work together (with a pull workflow rather than a push workflow). That's quite useful [1]
[1]: https://stackoverflow.com/questions/377213/git-serve-i-would... Further reading: https://git-scm.com/book/en/v1/Git-on-the-Server
> Yes, but you can also push/pull from a filesystem location. To be able to push it, it is simpler to init the repo with `git init --bare`.
Personally, I have build my very own and simple solution to sync encrypted files over the Internet using git with git-remote that uses filesystem location. The implementation evolved over time but initial idea[0] was to combine restic, pass and git with simple scripts to pull/push the git-remote repo (located in /tmp/repos) to S3 bucket via restic that takes care of upload, deduplication and encryption. Thanks to restic I also don't care much if I'd commit to stale (outdated) master branch, because it uses snapshots and it's quite easy to navigate between them.
[0]: Year-old PoC of encrypted repository share with B2 as a storage: https://gist.github.com/piotrkubisa/dece2fc71399efa56e2d0b8f...
Now I'm just annoyed that more teams I've been on haven't set this up!
I have two git repositories which somehow got into an inconsistent state: How can I reconcile changes in both repositories and resolve conflicts between mutable-metadata (branches, tags) in a sane way?
Alternatively you decide the one of the repositories is the primary one, set up a remote called `mirror` and set-up a `post-receive` hook to:
git push --mirror mirror
Tags: Don't have a process which can result in tags pushed into different places. It's a path to madness. Same applies to master/release branches.
Yes. This was the case I had in mind. Path to madness.
One serious question though: how do you deal with PRs when you do this? That's one area where it feels like things could be quite messy, especially if you have quite a few PRs going in throughout the day.
Having looked briefly into it now, git-dit does look promising in its approach. I'd be interested to hear from someone who had actually used it and bumped up against the limitations: https://github.com/neithernut/git-dit/blob/master/doc/datamo...
[1]: https://github.com/google/git-appraise
[2]: https://github.com/forgefed/forgefed
[3]: https://drewdevault.com/2018/07/23/Git-is-already-distribute...
If you have any discrepencies in between them, you'll need to merge locally of course.
For most people, it would be just a read-only copy. And the value of that is fairly small.
[timwolla@/s/xxx (master)]g remote show origin
* remote origin
Fetch URL: git@git.example.com:xxx.git
Push URL: git@git.example.com:xxx.git
Push URL: keybase://private/timwolla/xxx
HEAD branch: master
Remote branch:
master tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (up to date)> Note that the push URL and the fetch URL, even though they can be set differently, must still refer to the same place. What you pushed to the push URL should be what you would see if you immediately fetched from the fetch URL. If you are trying to fetch from one place (e.g. your upstream) and push to another (e.g. your publishing repository), use two separate remotes.
which seems to imply that weirdness might happen if the two happen to get out of sync, or if one (specifically, the one pointing to the repository you're fetching from) fails.
For something that may be a bit safer, I believe it's possible (but haven't tested) to have multiple values for branch.whatever.pushRemote-- that should do the same thing, and has the added bonus of making the secondary remote easily fetchable.
But why not just have different remote names other than the default of “origin”? Somebody else on the thread mentioned that it might be a bit complicated to clean things up after an outage on such a “multiplexed” remote.
What it does affect is the ability to do code reviews, work with issues, maybe even do releases. All the non-DVCS stuff.
Pushing to all three isn't that difficult. The hard part is reconciling after one of them suffers an outage or partition.
Perhaps what they really meant is that they couldn't get to stack overflow :-(
WTF is going on?
Mood: "Am I going crazy, or is it the world around me?" ~ https://fishbone.bandcamp.com/track/drunk-skitzo
Github built something that inarguably made development better.
- They host some huge number (tens of thousands? hundreds of thousands? millions?) of repositories for free.
- They built Github Pages for people to ship websites and host those for free.
- They built an intuitive flow for managing issues and pull requests, which is included with every repository for free.
- They integrate freely and openly with all sorts of third-party services, some of which compete directly with them. That's quite uncommon for a for-profit software platform of their size.
- They have given back immeasurably to the programming community, to the open source community, and to developers including sponsoring conferences, donating office space for events, etc.
- Their developers widely share their (i.e. done at/for Github) work and contribute upstream.
There is some argument that all of the above are self-serving for Github, which is proven false if you talk to a single developer at Github.
Do they adhere to every idealistic principle of FLOSS? No. But, honestly, who gives a crap?
So, to answer your question of WTF is going on: real, positive progress.
Linux kernel development doesn't shut down if kernel.org is down (heck, when kernel.org was completely pwned it only delayed the kernel release by a week or two). People still send each other emails and even if a maintainer is AWOL, other maintainers will pick patches and route them through their own trees. The only problem with the kernel development system is that it isn't easy to on-board people. But if we had a better UX for this system it would be far superior in every respect. There was a recent post by a kernel dev on how this might be improved by building on top of Secure Scuttlebutt and having a nice implementation on top[1].
[1]: https://people.kernel.org/monsieuricon/patches-carved-into-d...
Not sure why you had to lead with an irrelevant comment like that as the rest of your comment has interesting counter arguments.
I think Github did make development easier, but the fact that it's close source and now owned by Microsoft gives me an uneasy feeling.
With that being said, the only thing that does make it ok is the fact that Git is decentralized.
If Github was an svn host I definitely wouldn't have been ok with it or hosted anything FOSS on it.
The most sticky part of Github is the social network that lives in issues, pull requests, and repository permissions. This is all entirely centralized, and it is scary how heavily the community relies on it.
So in practice, results in just as closed of a system as SVN. And it doesn't matter if you're "ok with it". GH is where all the people and projects are. If you want to participate, you have no choice.
* Git isn't distributed. It's decentralized -- big difference. Distributed means there (usually) aren't any single points of failure. Decentralized means these are many single points of failure but failures are localized. Combine that with the reality that in most markets a "best option for most people" emerges and we get bigger and bigger points of failure.
* Microsoft has been trying to get into the socal networking game for a while now and found their match made in heaven with one based on software. Time will tell if the recent Microsoft will be a good steward -- so far it's been pretty good.
* Definitely a victory. The dominant VCS is open. More people than ever are contributing to OSS. It's easier than ever to share code.
[1] Github's on-prem offering is a completely different story and is very much a problem that it isn't OSS.
We normally can't complete tasks until they are code reviewed, so checking out locally would be the analogous of working offline in a CVCS
You don't have to wait for CI, just create a new branch and continue your work as though CI passed, and your PR was accepted. If that comes true you don't need to worry much, just rebase and continue.
On the other hand if you find out you needed to make changes - do those on the branch you made those changes, and finish your PR/CI cycle. Then go to the new branch you continued work on and rebase, and continue.
Is there something I am missing?
People get so caught up in their daily coding they forget about all the other fun coding they could be doing outside of work.
It's a shame.
Obviously it's a moot point for you now, but getting comfortable with how git remotes works is something that could pay dividends for you in the future.
Also, the additional effort to manage multiple remotes is entirely nontrivial.
Here is one of the top hits searching "git email patch"
https://thoughtbot.com/blog/send-a-patch-to-someone-using-gi...
As another alternative, git-format-patch/git-apply or git-bundle may be a quicker way of shipping changes around.
2. Send a branch in whatever file exchange service you use. (Also formatted as patch series)
3. Setup ad-hoc vpn (zerotier?) and setup remote via SSH.
4. Push the branch into a private repository on a different service (private gitlab?)
5. Spin up a free t2.micro on AWS and push the branch there.
There are lots of options.
If you need access control without VPNs or whatnot (or if you simply want something closer to the github workflow), there are other services similar to github that can almost certainly provide what you need, such as gitlab.
> "Also, the additional effort to manage multiple remotes is entirely nontrivial."
With practice, I'd say it asymptotically approaches trivial.
dig raw.githubusercontent.com +noall +answer 0 < 12:54:48
; <<>> DiG 9.14.3 <<>> raw.githubusercontent.com +noall +answer
;; global options: +cmd
raw.githubusercontent.com. 6 IN CNAME github.map.fastly.net.
github.map.fastly.net. 19 IN A 151.101.20.133
And now I come back here and see this.. :D
I'm looking at all of you CocoaPods, SwiftPM, npm, Cargo, vgo.
It's true that most developers rely on GitHub for pushing to the registry, but there are a handful that use GitLab or self host that would be totally unaffected.
sigh
Not to mention, if you get the self-hosted route you can use Gerrit, which is still miles better for code review than GitHub, Gitlab, bitbucket and co.
apt install git-all
is enough to host your own git server. Put it behind a firewall to limit access and use standard linux users with ssh keys for access control if you don't need anything fancy. For small companies I'm not sure you need anything else. Of course if you need different levels of access etc then you'll need more sophisticated tools, but many people won't.
Code review I do using local tools (the editor) face to face, again not sure you need an online service for that unless you're a larger company with lots of developers coordinating (in which case it becomes pretty essential).
I mostly like online code review services because they offer an audit trail and semantic history that's easier to navigate than email. And of course, to let CI automation check tests, coverage and lint. Not because I don't trust my coworkers, but because otherwise I would forget to run tests and lint myself.
For lots of small projects though, it's perhaps not as necessary as people think. I run tests and linting locally on save and don't really use the code review/CI features of online hosts much. That won't suit everyone of course, but it is one possible path.
1) dependent reviews/change requests. I will work on some feature, submit it for review as one CR, and then I can immediately start working on a feature that depends on that. When I submit this one for review, it will be always shown as dependent on the first, and show a diff against master after the first is merged. This also means you can split large changes into multiple CRs, have them reviewed (possibly independently), then submit them all at once. It makes changes across large repos fantastically easy.
2) very powerful rule engine for approvals. It's based on Prolog, and basically allows you to define arbitrary, turing complete rules on what labels added by whom must be present on a CR for it to be submittable. Using the 'owners' plugin, you can also make it depend on OWNER files that define ownership in subtrees of the repository. This can lend to rules like 'product A must be approved by an owner of A but cannot be self-approved; in addition, someone who is fluent in the languages used must approve it, but that can be self-approval'.
Without those two working in Git monorepos is painful. And since I like monorepos for other reasons (like ease of deployment and testing), I like Gerrit, too :).
It also offers, in my opinion, a much better UI for actually reading and commenting on code. High contrast, fast keyboard navigation, marking of files as reviewed and a very readable history of patchsets, comments, approvals, etc.
The learning curve is much steeper than a GitHub PR, as it's a somewhat weird abstraction (CR/patchset vs git commit/branch), but in my opinion it's worth it. I guess it's my general tendency to use less beginner friendly but more powerful tools. ^^
Finally, I'm a big fan of the various labels that are common. +2 Code Review means I reviewed the code, +1 Verified means that I ran it and it worked. Those are different things and having to have both makes the responsibility clear, even if the author is adding +1 Verified.
This is really nice in Gerrit. On GitHub you can simulate this by changing the base of the PR yourself, but it's not as smooth experience.
Have you looked at the GitHub incident history? https://www.githubstatus.com/history
Why the hell would you think that after the Microsoft acquisition?
(This creates a bare repo on your work computer, meaning there's no associated working directory -- you'd probably want to add that same repo as a remote from whichever existing repository on your work computer you have. The bare repo, in this scenario, is just a means of passing commits from your laptop to your work computer in a Git semantically-meaningful way.)
As with most things, it'll start great, but a lot of those people will be in tears in a few weeks.
Great power; great responsibility.
Here's an alternate setup that doesn't use a bare repo. It does require some git hygiene/discipline though.
Setup: Desktop: git config --local receive.denyCurrentBranch updateInstead This will let the laptop push the desktop, updating its files, as long as the desktop doesn't have uncommited things (aka working dir is clean). On the laptop: git remote add desktop...
Working with it: Desktop: commit everything Laptop: commit everything git pull --rebase desktop git push desktop (assuming there are no issues w/ the pull).
Not saying either workflow is better, merely providing an alternative.
imo the reason git took over has some to do with being unopinionated about workflow - there's some tooling, but whatever workflow is managable with that tooling is "supported" - as long as the team can agree to use said workflow.
https://unix.stackexchange.com/questions/10026/how-can-i-bes...
For example https://medium.com/@alexberegszaszi/mango-git-completely-dec...
You probably mean centralized internet based services.
I worked on our devops systems for a while. Every `git clone` had to have multiple retries, and even then there were multi-minute outages multiple times a month that caused things to turn red and caused distrust in our CI pipeline.
I tried hard to get my company to not rely on github as part of our CI process (as others in these comments indicate) but that's an expensive proposition for many - similar to relying on other third-party cdns like dockerhub, npmjs.org, etc.
That being said, I am, and probably will still be a massive GitHub user, because I value its community aspect. But I also now have unlimited, truly-private Git hosting for peanuts (cheap DigitalOcean server), which is always nice.
On server:
mkdir -p ~/repos/foo.git
cd ~/repos/foo.git
git --bare init
git remote add backup ssh://user@server/~/repos/foo.git
git push --all backup
edit: looked up commands, made a couple minor tweaks.
note: this doesn't include any kind of LFS support if you're needing it, consider a more complete install, unsure on gitea or gitlab etc.
These steps will cost you < 15 minutes but you could end up with significant savings.
1. make an account on gitlab
2. create a new repo
3. configure it to mirror your existing GH repo. The mirror will periodically pull from GH without any interaction from you, so you always have another option.
To be clear, I don't have any opinions about the operations of Github vs Gitlab. I have no idea which has superior procedures or equipment to avoid downtime. But I am sure that this small effort to diversify is worthwhile.
EDIT: wowsers downvotes for practical advice? Downvoters, please chime in on how my advice could be improved.
The problem is usually not that people loose access to their code (you usually have a local cache), it's that you can't open or review PRs, and that you can't trigger CI builds.
Using _both_ GitHub and GitLab simultaniously for more advanced workflows is a big hassle.
1) It's really, really, easy to setup a second mirror that you can switch your CI/CD processes to in the event of downtime in order to avoid being affected
2) Unless you don't vendor your builds and some of your third party dependencies rely on Github as their sole distribution mechanism in which case you are SOL
I wonder what the retention incentive period was for Github employees; it's often a year, and the Github acquisition was mid-2019...
Is this going to be a PR stunt like hotmail - they're going to say the fix was migrating to Azure or upgraded from Linux to Windows 10.
Because one of the main points of Git was to provide a "distributed" version control system.
Meaning that you are probably using it wrong if your entire company completely halts to a stop just because Github/Gitlab/Bitbucket are down.
Maybe what we need is a service that automatically consolidates them for you so your repos are always online?
What is the "them" you're referring to? Perhaps you're talking about consolidating multiple hosting services? This seems dangerous. What happens if Susan can access GitHub but not Gitlab and the reverse for Thiago. Susan pushes updates to GitHub, Thiago pushes updates to Gitlab and now you have to figure out how to reconcile the too.
Of course you have a similar problem if both commit different changes locally, but in that case it's generally agreed upon that the hosting service holds the version of record and that developers should reconcile changes against this.
That's a variant of "did you even read the article", which the guidelines ask you not to do (see https://news.ycombinator.com/newsguidelines.html). That's partly because it's a putdown, but mostly because there's no information in it. Readers who know less than you do are here to learn, so a better version of this comment would share some of what you know.
There you go: fully IPFS based version control and collaboration. https://radicle.xyz
Pretty obvious someone didn't like one of my other comments and then proceeded to downvote the others they could since I commented in multiple threads at the same time. Why is this nonsense allowed? It would be easy to detect.
Also, if you've been here long enough to get sick of anything you should have learned that dumb downvotes happen, they mostly get reversed over time, and nothing good comes of reacting to them at all, much less throwing a expletive-laden fit about them.