Hacker News Reader

Comment by everfrustrated | Hacker News Reader

Enterprise customers, remember to email your sales rep and ask for them to report on their contracted uptime with you that you are allowed to do as per contract. They wont do this unless you ask hoping you don't notice the outages. It creates lots of internal pain - they have no automation internally for reporting on this.

This is the only way anything will ever change. GitHub is _easily_ the most unreliable SaaS product. There's not a week whereby we aren't affected by an outage. Their reputation is mud.

View on HN

> GitHub is _easily_ the most unreliable SaaS product.

Some of us are stuck using Atlassian and BitBucket and it is by far worse in every way.

Agree, atlassian products are much worse in terms of reliability.

We use Jira. It's a horrendous product, but I cannot remember the last time it was down, unlike Github.

I use self-hosted jira, it's a great product, but I have full control over my teams tasks and workflows and as a tiny team we make them work for us (subject, description, comments, occasional linking to other tickets, assigned to, and status of "open", "blocked" or "done")

Most of the problems I hear about are micromanaging product managers. That's not the fault of the tool itself per-se.

Amen, BitBucket outages are regular and showstopping. Self hosting is genuinely more reliable.

Noooo!! It's impossible to get three nines uptime when you self host something on a single raspberry pi, I read it on the internet so it must be true.

Ah, you clearly haven’t worked with the self hosting teams that I have.

I hope I never again have to explain to someone that you can’t just “restore the code from the weekly database backup” because the code is in the file system they just osmosed.

That's not the fault of a hosting team though.. that's the fault of a hiring manager, ultimately

It's also _by_far_ the cheapest, and most git-work can be done offline, so while it's inconvenient, it's not stopping all operations here.

Depends how intergrated you are. If you're using it as a code repository fine, if you're tying your workflows into it with pull requests, actions, maybe a third party CI which ties to it, and use it as part of operations then it's a major problem

I just approved a PR which added a user to one of our AWS accounts for example, if github is down then that PR can't be approved, the update can't work and the user can't access that account

I mean you can always work around it with the correct permissions right.

I've only had good experiences with Gitlab.

Funny, I've had the worst experience with gitlab. Bitbucket is the best experience I've had so far

I used GitHub for years in the tech industry, then went into the games industry and used BitBucket and hated it, thought it was such a downgrade. Now I'm back in the tech industry and using GitHub, and I miss BitBucket.

Is games not a part of tech?

Of course, but there are some oddities in tool use compared to other industries. At my job we use Perforce for version control for example, which I think is more common in the game industry than other solutions for whatever reason. Naturally everyone here hates it.

But not everybody here has to try and manage many GB or even TB of assets in their VCS. I wager game company build/dev engineers know what they are doing in picking Perforce.

It is a _decision_ to put those assets in the same VCS as your source code. It's not one you are required to make.

...maybe, but there a good reason not to make that decision, assuming you have a VCS that can manage large files?

I am very much not in this industry, but it seems to me that if the assets and code depend on one another, you'd want to keep them together.

> Perforce for version control for example, which I think is more common in the game industry than other solutions for whatever reason.

The last game I worked on was like 80gb built. The perforce depot was many terabytes large, not something you want to have on every person's workstation. Games companies use Perforce for a very good reason.

I liked the UI of BitBucket more. Stuff I accessed frequently like commits and branches were tabs across the top, easy to reach from any page. Branch dropdown sort order was by most recently updated unlike GitHub where I have to search for it. Easy to diff files. This was like 4 or 5 years ago though, maybe it has gone through modernization/enshittification. GitHub feels a bit fragmented and it tries to be more performant by virtualizing some things but while BitBucket in some sense was more rudimentary and showed it all (bogging my machine down some), it allowed me to CTRL + F easily with more confidence whereas with virtualization I've had issues with it finding things and I couldn't 100% trust it.

> then went into the games industry and used BitBucket and hated it

At least it wasn't Perforce?

I've worked at two game companies and both used both of them. And I went to grad school for game design and used Perforce. So yeah I've ... had the experience.

Bitbucket is butt

Starting to pretty much desire tools that can do nothing but are just fast at their core competency.

Looking at PRs in github and then when toggling to the "files" tab it chocking up or being like "I don't want to display this file because its more than 100 lines" is like wtf you're whole point is to show me modified files.

Like the Unix philosophy, but for businesses.

Come to think of it, this might be great advice for life in general: do one thing very well, and be modular (aka play well with others)

I wish the world worked this way, but I don't think it does, especially in tech. If you "do one thing well", the cloud hyperscalers will use their billions to copy whatever that is, and add your "one good thing" into their bundled subscriptions or cloud plans. At which point, any rational CTO will go "why should we pay for this, when we're already getting it via AWS/O365/whatever, and with better integration with our existing tooling to boot?"

I don't think "do one thing well" can succeed in this world, which is why Atlassian, Dropbox, etc. keep on launching things like office suites even though that makes no sense considering their core competencies. It's the only way not be streamrolled by FAANG.

It works with Keiretsu. Ive long thought that this is how Europe should play catch up and build its own equivalent(s) of AWS, GCP and Azure.

I've seen it mentioned in the context of smaller countries competing with the big guys using industrial policy, and one example mentioned in that article (I don't have a link - it was long ago) they specifically mention Sweden and Volvo.

I'm about to sound crazy.

Github's problem is that it isn't a SPA. It is a massive Ruby on rails project that is all server-rendered. Everything you do needs to be synchronous and almost everything requires a reload. A react or angular app with great restraint would be dramatically faster at all of this as viewing a file is just an API call - not a page reload. They are stuck with their hands tied as loading large data would cause the whole page load to be delayed - thus silly limits.

Many things should not be webapps... but an app on the web like this...probably should.

You can have parts of the web app rendered on the client, and still keep the rest of the app the same. Rewrite the diffs and previews, keep the rest as-is.

There is no excuse for possibly the most used feature of Github to suck so badly.

> Everything you do needs to be synchronous and almost everything requires a reload.

this is pretty incorrect, you may want to look into the concept of "partials" in SSR. maybe you meant everything requires a roundtrip ? but SPA would not solve most of the roundtrips necessary in github given many interactions in the github app require authn/authz checks.

would you care getting into more details ?

Also, 'old' github was known to be very fast an reliable and was indeed a ruby on rails SSR app. Since a few years ago github started to introduce react and more client side logic and it correlates with more issues and more slowness in the frontend. It only correlates, but still.

I am increasingly in favor of inspecting/diffing branches from MRs locally.

We use so many cloud services and SaaS products that it will be quite a shock how fast systems can be. You don't even need to use a local service, just hook up a minimal server somewhere that pulls from slow services here and there. Technically cloud, but not as shitty.

I think modern wait times are crushing for productivity, it is really demotivating and wears you down. Either just skip to another task and get overwhelmed by context switches or you wait and degenerate to a non-thinking troglodyte.

Grass is always greener on the other side.

Isn't Sourcetree an Atlassian product? I used that for a while when my company outlawed Github and it seemed to do a good job.

Atlassian Jira and Confluence rock. Can't recall ever using BitBucket but Atlassians's products are top notch.

I used to maintain a self hosted instance of BitBucket and the user experience of it was actually very nice. We shut it down when Atlassian deprecated the self-hosted licenses. Moving to GitHub and GitHub Actions felt like a downgrade in more than a few ways

Two customers of mine have been using Bitbucket and other Atlassian products. I remember a problem a few months ago, nothing else. Maybe I've been lucky, no accesses during the outages.

Bitbucket is quite nice actually. It's got very little bloat and just... works?

Companies should automate this. Write their own outage monitoring, feed the results, plus the cumbersome format you have to send to the provider, into an LLM, have it spit out an email requesting SLA credits or whatever the contract specifies.

Probably not worth it for low cost services, but if you’re paying GitHub $x millions per year, maybe it is.

They intentionally underreport outages. Everybody does. When your performance metrics for your customers, managers, and individual contributors all include uptime, what you get isn't better uptime but lies about uptime.

Some customers of my product, StatusGator, do this with our API. They can extract the outage data -- including the time when we detect the outage before its acknowledged. And then use that to get SLA credits.

Why would I trust you to report

Its great that your specific product does this, but as a whole I have to monitor the service separately to keep you honest (well not you specifically, I'm sure you are honest and do as much as you can to be honest, but not every company is), and of course to monitor the problems I have which you don't detect.

Gotta get a monitor for the monitor, then a manger for the monitors, and a manager to manage the managers of monitors.

I think that was sarcasm

Obviously you should use a SaaS for SaaS uptime monitoring. No need to build this yourself.

You can also self-host something like Prometheus or Uptime-Kuma.

> GitHub is _easily_ the most unreliable SaaS product.

I'm no fan of Microsoft either but when you say ridiculous things it's hard to take you seriously.

Is it that much hyperbole, assuming we ignore small fry SaaS? I've personally noticed at least three outages in the last few weeks alone.

Sure it's less impactful than some services since you don't need access to the website at all times, but the reliability is still really bad.

> Is it that much hyperbole, assuming we ignore small fry SaaS?

Well there's Claude for starters.

There's thousands of enterprise SaaS products. I seriously doubt the OP is referencing any grand audit of them or could even list 20. Yet one they dislike is "easily" the worst. They write like a child.

All this does is make some poor offshore schmuck toil away for hours trying to get this data.

If companies begin to _cancel_ their contracts with MSFT/GH because of a breach of SLAs, then maybe conditions will improve.

Reality: companies locked into multi year deals with MSFT including a MSFT-shit suite and windows licenses.

Migrating away from it will be expensive. MS knows this. Thus the reason why nothing will change.

The way you change this is not by pressuring Microsoft to be better or less of a shitty company.

The way that you change this is by switching to a different forge host, or by self-hosting Gitea.

I do so, and it’s simple and painless and cheap, and this quarter my uptime is better than this multinational’s.

this, but unfortunately many Fortune 500 are so deep in the MSFT stack. It would take many years to migrate off.

Former githubber anon posting a non main acct

You all should be running a small private git

Most teams could get buy with something like a tiny instance with snapshots every commit

AWS has too much skin in the game to be as unreliable as they used to be.

There's zero reason for a startup to use all these services anymore. The only reason they ever existed was big government manipulation of the labor market through ZIRP

It's far more set n forget to self host git than github will be

Customers should also then compare what the sales guy is telling you to your own metrics, and see how far off they are.

I've had much better luck with GitHub than GitLab and whatever Atlassian is calling their source control these days.

Still sounds like good advice though.

Reply on news.ycombinator.com