[alias]
thrust = "!f() { until git push $@; do sleep 0.5; done; }; f"
I've done a few pushes so far, and found that it's going through in <10 tries or so. [alias]
thrust = "!f() { until git push $@; do sleep 0.5; done; }; f"
I've done a few pushes so far, and found that it's going through in <10 tries or so. # Retries a command a with backoff.
#
# The retry count is given by ATTEMPTS (default 100), the
# initial backoff timeout is given by TIMEOUT in seconds
# (default 5.)
#
# Successive backoffs increase the timeout by ~33%.
#
# Beware of set -e killing your whole script!
function try_till_success {
local max_attempts=${ATTEMPTS-100}
local timeout=${TIMEOUT-5}
local attempt=0
local exitCode=0
while [[ $attempt < $max_attempts ]]
do
"$@"
exitCode=$?
if [[ $exitCode == 0 ]]
then
break
fi
echo "Failure! Retrying in $timeout.." 1>&2
sleep $timeout
attempt=$(( attempt + 1 ))
timeout=$(( timeout * 40 / 30 ))
done
if [[ $exitCode != 0 ]]
then
echo "You've failed me for the last time! ($@)" 1>&2
fi
return $exitCode
}>Time for some manual DoS
I would argue GitHub has done more for societal good than most tech ventures, by the way.
At the end of the day - our companies also have people that rely on our software working in order to do a lot of societal good.
"Having problems" in this world (any kind, not only due to the github scale!) is something that happens - we are not perfect and we work on an incredible amount of layers of complexity.
It is sufficient to actually touch production code on a daily basis to see that it can happen to the best, with the best observability systems or processes. The key is avoiding blaming, and understanding iteratively how to fix the problems underneath (faster recovery, detection time, and so on).
And the crazy thing is you see people on HN demanding that some one person side project/SaaS has to be at 100% uptime with multiple failovers, automatic scaling, etc. etc. There is such an emphasis on scalability on HN and yet... you just brush that all away because "software is tough." Yeah, no shit. Poor Github. They are also Microsoft now. One of the wealthiest corporations in the entire world. And people are paying Github. This isn't Twitter fail whale we're talking about.
I'm not one of those people. I may have been when I was much more inexperienced.
Software is hard. Full stop. Organizational politics, engineering culture, business / tech alignment are all hard. Distributed systems are hard.
> Yeah, no shit. Poor Github. They are also Microsoft now. One of the wealthiest corporations in the entire world. And people are paying Github. This isn't Twitter fail whale we're talking about.
I may have also thought this when I was much more inexperienced. This isn't a resource problem. Even a small startup, when they start having failures due to scale from growth, it's not a money problem. Throwing money at this doesn't make it go away.
By the way, the Twitter fail whale impacted paying customers (advertisers).
It doesn't change the fact that the company has absolutely crap dev culture which seems to put features first and foremost, at the expense of everything else. There are products with even more complexity that don't fall over and die almost every single day. It's just not funny anymore. Facebook is pretty complex, it had major issues like this one, what, once in its entire life?
I don't remember Google Search (or other Google products) ever not answering my queries, and I've been using it for about 18 years.
And so on. I reckon it's because those companies have strong engineering culture (Google certainly does, at least), and this one doesn't.
Anyway, you shouldn't fire someone for causing bugs in production since it indicates a systemic failure of all the checks that should come before the bug is deployed. Even if you can trace the root cause to one person, it would be counterproductive to fire them, because now they've made the mistake they probably won't make it again. Whereas their replacement doesn't have the same wisdom.