GitHub Is Down
githubstatus.com
githubstatus.com
Not for me, because I pay for the service, and I'm extremely happy with the value it provides for the price. I don't like downtime either, and I think GH needs to work on their stability, but I've also worked in companies that hosted all their source control on prem, and it had plenty of downtime, too.
> We also see the status come up a lot because folks have centralized on a service for a distributed version control system and mirrors have become increasingly less common.
At least if it actually went down for a long time you could just push your repos from whoever has then checked out to a new mirror and get going again.
From those experiences I wouldn't recommend Github anymore because I really question the benefit of having a blackbox-as-a-service you're fully locked into kinda randomly failing, IIRC correctly Actions was down 2y ago for 1-2 days which is like, a lot these days.
It's no wonder why going all in on Github makes absolutely no sense. Like I said years ago, it is better to just self-host or mirror your repository to GitHub.
Blender surely did choose wisely [2] and perhaps this is the time to self-host your repositories and not 'centralize everything to GitHub' [3]
[0] https://www.githubstatus.com/history
[1] https://news.ycombinator.com/item?id=32752965
for i in (seq 10); git clone https://me:api_key@github.com/… foo; rm -rf foo; sleep 5; end
I wish Circle CI would support Gitea, because not setting up the CI system and just configuring the pipeline would save a lot of time.
Everything is currently marked as "degraded", but in reality the entire website is returning 500s.
I don’t see how any culture or geographical location can possibly make deploying outside of work hours reasonable.
Obviously that points to larger problems though.
Does it, though?
Given that the readership of this site is ostensibly hackers and hustlers, including plenty of people with side-businesses from which they are at least trying to earn money, is it really so odd that somebody might be trying to do a deploy at an "unusual" time?
I'm as guilty of this as anyone else in the past, although I try to restrain myself now, but I'm constantly amazed at people who want to wade in with a hot take on practices and processes off the back of next to no information.
Here are reasons that I can think of for deploying at an unusual hour:
- As you suggested, production is broken
- Plenty of companies exist whose busiest times are weekends, or where events happen at weekends for which systems need to be available
- It's a side business, so it gets worked on outside of normal business hours
- It's an important source of income that will be disrupted (regardless of the size of the business, or whether it's main or side revenue)
- It's a startup, and people who work there are focussed on making the business a success at all costs (different points of view on whether that's a valid life choice but, for me, grown adults should be allowed to make their own choices without facing undue criticism)
- The world has mostly normalised on a Monday - Friday work week these days but there are still countries that have a one-day weekend, or where the weekend is split (I'm not sure if Thursday/Friday is still a norm in any Muslim countries these days). Point being, Saturday might be a business day or an important trading day
- The company could be a total shambles with poor processes and practices run by clueless management and leadership (but we don't know that and we shouldn't assume it)
- I'm sure there are plenty of other reasons for deploying late on a Friday
But, no, GP decided to lead with, "It’s late night Friday. Don’t." which I think is pretty arrogant, and to be discouraged. As I pointed out, we really don't know what's going on here, and deploying on a Friday might be a perfectly valid thing to do.
This is a reason to not do Friday deploys. It's always going to be harder to get anything done on weekends if stuff goes wrong, you don't want to add in deploys to that. For example, contacting a third party provider will be harder on a weekend.
I can see the reasons for a Friday deploy - if it must be done or it's just low stakes (which applies to a lot of projects where everyone can just go home and fix issues on Monday with some mild grumbling) then it's fine but otherwise I do think avoiding deployments on a Friday is a very good rule. They certainly shouldn't be routine unless you've got very good justifications.
I’m going to go based on the information posted. If someone posts that they’re deploying on Friday with no other information, I’m going to tell them the sensible thing, to not do it. I’m not going to come up with a million and one excuses about why they could be justified in doing it, I’m going to tell them not to unless they provide the justification. Sorry, not sorry.
It’s not perfectly valid for the vast majority of cases. You coming up with (frankly, extreme, and in most cases far from extenuating) edge cases in no way invalidates my point. Deploying on what is generally considered the end of a work week, without extenuating circumstances described, is going to elicit the same reaction. Because it’s the appropriate reaction to someone doing a risky thing after hours at the end of a week.
Your examples hardly even justify deploying outside of work hours. A scrappy startup or side business should not employ the shittiest of practices this early as it will only lead to these shitty practices being ingrained down the line. Almost every culture has shifted to the Western system and at best has Friday/Saturday as weekend, and even if weekends are the most important time for a business, it does not somehow justify deploying right before the start of that busy time when most people will not be working. There’s many things you should be doing at 9PM before your busiest time of the week, and deploying ain’t it.
> I'm not sure if Thursday/Friday is still a norm in any Muslim countries these days
It’s not, but you consider deploying on a weekend to be more sensible? Ok, sure Jan.
> But, no, GP decided to lead with, "It’s late night Friday. Don’t." which I think is pretty arrogant, and to be discouraged. As I pointed out, we really don't know what's going on here, and deploying on a Friday might be a perfectly valid thing to do.
Quite frankly, I don’t give a shit if a random person on the internet thinks it’s arrogant. Go off sis, you do you. I’m going to continue telling someone that wants to deploy on a Friday not to.
Anyone knows what service is called green.
So far in February there have been 14 incidents. Today is February 17. If you do the math, 14 ÷ 17 = a lot of fucking downtime.
For example, issues with repos themselves (pulling or pushing code) or Actions (which many folks use for their deployment pipelines) would be a lot worse than issues with Codespaces, in general.
You cannot keep developing without Codespaces if you use them in the first place.
GitHub is definitely down more often in the last couple of years though. It's noticeable. Hope they figure it out.
Things like the database or random microservices that implement the death star architecture.