Hacker News Reader

Comment by opiniateddev | Hacker News Reader

Given the recent history of outages, I wonder how many are seriously considering moving off GitHub for anything other than code hosting. I mean actions / workflows.

View on HN

To offer a data point (not a tribalist argument): GitHub Actions being down this often is still less expensive than the cost of switching everything somewhere else. Would rather have my team walk away from the PRs and do other tasks than have engineering effort, meetings, design pages, scheduling, etc. for switching over.

This is annoying and I'm here because it's down. But it would have to be far worse to come close to actually being worth changing.

A case study in vendor lock-in.

Pretty much, eh?

If I could right now:

1. go sign-up elsewhere 2. Log into GitHub and point Actions to that new host 3. All my actions files immediately worked without question

I'd probably give it a spin and make a wiki page explaining how to swap back and forth. No meetings. No design issues. No scheduling. Just a flip switch on who to pay for computers.

That's why GHA makes you write these weird ridiculous YAML files instead of shell scripts.

For what it's worth, we switched some of our actions to a self-hosted Woodpecker instance, and although there were a few kinks to iron out, it works better overall (for example, because of better caching on that single instance, our docker images build faster).

In my experience moving from GH to Forgejo was trivial, largely because the actions are mostly compatible and importing the repos can be automated. The amount of time it took was definitely less than the 5-8 hours of downtime that GHA had yesterday.

I’m intensely grateful that despite rumblings, the company I contract to hasn’t switched from self-hosted Jenkins to GHA. Hopefully the steady drumbeat of failure will keep it that way.

If that’s how you feel, make sure you share that feedback with the deciders!

We've started writing some new workflows against argo-workflows for the last like 9months or so. Especially for use cases where the actions are like self-service type automation.

I think people were so excited to move away from jenkins to something 'managed' just because of how much a dinosaur jenkins is and how much a pain in the ass it is to upgrade it... but now we are seeing how managed can bite you in the ass if the manager is incompetent.

GHA still beats Circle CI or trying to run your own Jenkins - even with all the downtime. I'd move in a heartbeat if there were good alternatives.

I think I agree if you don't have someone dedicated to running and maintaining Jenkins. If you have someone at your org that knows what they are doing though, I'd still take Jenkins over GHA any day.

gitlab self hosted though? no outage. i remember equivalent functionality at my last job taking this path

That's significantly more difficult to set up and it cost money. Github's big issues that it's free, Microsoft doesn't want to allocate enough budget to keep the thing running properly.

But for people who either don't pay anything at all or phenomenal amount one 9 of up time is all you need.

If you're actually trying to run a business I guess you can call and gitlab and get an Enterprise contract

That's not just moving actions though, that's moving _everything_ and taking on the maintenance of running it yourself.

We've been working on this at Depot[0]. We built the internals of Depot CI as a general-purpose workflow engine that understands GitHub Actions as an input (with plans to extend) - we've been working to make Actions runners more performant and more reliable for a while, but eventually you hit a ceiling if you don't also control the control plane.

[0] https://depot.dev

Our deployments are triggered by GitHub Actions, and it is a pain to deploy without it (specifically collecting the credentials and putting them into variables the shell will read).

This is multiple times this month that this has been a problem.

Has GitHub completed it's internal migration to Azure yet? Or is it still ongoing? None of our devs want to switch away from GH, but we will have to at this point.

Mitchell Hashimoto had a blog post a few months ago talking about him moving off of GitHub: https://mitchellh.com/writing/ghostty-leaving-github

This is a man who's spent a significant portion of every day for the last 15 years on GitHub.

Ghostty is still there though? https://github.com/ghostty-org/ghostty doesn't seem like a serious effort. And:

>My personal projects and other work will remain on GitHub for now.

People always talk about "leaving GitHub" but it's all talk.

Did he say where he was moving to? Couldn't find it myself.

What would you guys recommend as an alternative? Both hosted and/or self hosted

Here's what I did.

Step one - migrate my build workflows to Docker.

My Github actions are now basically: "checkout / set env vars from secrets / docker-compose builder run make".

I used large machine runners to run full the Docker (Podman actually) on Github first to avoid dealing with docker-in-docker complications. This step also provided some very nice robustness advantages, as I can now trigger deployments from my laptop if needed.

Step two:

Migrate to self-hosted runners. I used my former homelab server to set up a build machine. It has 16Tb of fast NVMe SSDs and thanks to Podman container layer caching, my entire lint workflow now takes 30 seconds. Faster than just one "npm install" on Github before.

And Github's self-hosted runners are actually surprisingly easy to set up and use. They are also somewhat more robust.

Step three:

Swap Github for something else.

WoodpeckerCI for DIY, Drone.io for paid support. Self-hosted, OAuth login, repo-permission-based authorization, container-native design, easy web UI, optional RDBMS, supports many platforms. All the core features needed for scalable CI, deploy as few/many as you want for multiple teams, small enough to easily run on one box for one repo. Nothing else is as simple, easy, powerful, compatible.

We have set up a new enterprise and are migrating to GHE. For us this was a no-brainer; we get our own isolated environment, Enterprise Managed Users with Entra OIDC provisioning and Github Copilot in EU data residency. https://eu.githubstatus.com/posts/dashboard looks pretty good to me. We will still keep our github.com enterprise around for public repositories. I’m genuinely curious to know why a company would prefer github.com over GHE?

I wasn't directly involved, but working in a couple of shops that used GH, I think they really steered people away from GHE, my impression was it's a product they don't really want to support - I vaguely recall hearing that upgrades were very painful on GHE, and the scaling / failover story was not great. Can anyone else confirm?

To clarify, we are on Github Enterprise Cloud, not Server. The option for enterprise managed users with your own ghe.com subdomain and data residency options is still prominently shown in the enterprise signup flow. https://docs.github.com/en/enterprise-cloud@latest/enterpris...

We self-host gitlab and it’s great. It’s not cheap, but it’s also never gone down.

Forgejo seems decent but I've only scratched the surface.

Forgejo selfhosted for a few months now. Its all good and cost nothing.

Codeberg ironed out a lot of issues in Forgejo

how hard could it be these days for a mid to large size eng company to have their own gitlab/hub type solution hosted in aws

I seriously want to. I'm just waiting for the day where I have enough extra energy to simply bite the bullet and do it.

Never mind. It's happening now! Migrating to self hosted Gitea for work. Super excited!

I would conjecture that the long tail of GH users are not dependent on any of the CI stuff in the first place.

To what? That’s kind of the problem.

My company moved to GH Actions 6 months ago even after I pushed back with a "Are you sure given their reliability issues of late?"

Now I'm stuck twiddling my thumbs with PR checks stuck/failing...

Because the exec played golf with a Microsoft exec. That's what really makes decisions.

Until someone makes a better PR experience you guys are stuck with GitHub.

Nobody cares about ATProto or whether your commits are a damn NFT or some bs just literally improve upon the experience.

That’s it.

It’s as if no company is focusing on the product experience or anybody’s experience anymore. It’s all ooo look what I got I got this I can do that too me me me but nobody will ever buy that.

Say what you want about huge companies like Microsoft or Walmart but they spend a lot of energy understanding the human experience to sell products and less on their own perceived self-aggrandizement.

GitHub is the best version control online and it’s not even close.

Forgejo has a better PR UI. Github has degraded so much that its UI can't even _show_ _the_ _fucking_ _diffs_ without clicking on "Load Diffs".

Github is just the laziest default. It's not _terrible_, but it's also not great.

I hear good things from people who've used sourcehut's "email a patch" model, believe it or not.

Also GitHub just sucks.

Reply on news.ycombinator.com