HNHacker News
TopNewBestAskShowJobs

llama052

587 karma · joined August 17, 2014

Infrastructure Engineer / SRE ping me at aG5AYWx1Y2FzLm1l (base64)
submissionscomments
llama052··on Atlassian to cut roughly 1,600 jobs in pivot to AI
You can’t copy and paste markdown, nor can you use it in the native way in confluence. They have third party paid plugins that can do it.
llama052··on Atlassian to cut roughly 1,600 jobs in pivot to AI
I'd argue the opposite. The thing is so bloated and the simplest of things seem to be so hard. Import markdown in confluence? Nope not natively. Add an issue to the board? better go to the one workflow to do it and not in the actual ticket.
llama052··on Cloudflare outage on February 20, 2026
I’ll take clarity and actual RCAs than Microsoft’s approach of not notifying customers and keeping their status page green until enough people notice.

One thing I do appreciate about cloudflare is their actual use of their status page. That’s not to say these outages are okay. They aren’t. However I’m pretty confident in saying that a lot of providers would have a big paper trail of outages if they were more honest to the same degree or more so than cloudflare. At least from what I’ve noticed, especially this year.

llama052··on Microsoft open-sources LiteBox, a security-focused library OS
Both can be true.
llama052··on Microsoft's Copilot chatbot is running into problems
It feels like that's the entire MO of the Azure platform as well. Make a minimum viable product and then get adoption by selling at all costs, despite the products edges.
llama052··on GitHub experience various partial-outages/degradations
We've ran into that issue as well, ended up having to move regions entirely because nothing was changing in the current region. I believe it was westus1 at the time. It's a ton of fun to migrate everything over!

That’s was years ago, wild to see they have the same issues.

llama052··on GitHub experience various partial-outages/degradations
It's awful. Any other service in Azure that relies on the core systems seems to have issues trying to depend on it, I feel for those internal teams.

Ran into an issue upgrading an AKS cluster last week. It completely stalled and broke the entire cluster in a way where our hands were tied as we can't see the control plane at all...

I submit a severity A ticket and 5 hours later I get told there was a known issue with the latest VM image that would create issues with the control plane leaving any cluster that was updated in that window to essentially kill itself and require manual intervention. Did they notify anyone? Nope, did they stop anyone from killing their own clusters. Nope.

It seems like every time I'm forced to touch the Azure environment I'm basically playing Russian roulette hoping that something's not broken on the backend.

llama052··on GitHub experience various partial-outages/degradations
Sadly Github moving more into Azure will expose the fragility of the cloud platform as a whole. We've been working around these rough edges for years. Maybe it will make someone wake up, but I don't think they have any motivation to.
llama052··on GitHub Actions Have "Major Outage"
Looks like Azure as a platform just killed the ability for VM scale operations, due to a change on a storage account ACL that hosted VM extensions. Wow... We noticed when github actions went down, then our self hosted runners because we can't scale anymore.

Information

Active - Virtual Machines and dependent services - Service management issues in multiple regions

Impact statement: As early as 19:46 UTC on 2 February 2026, we are aware of an ongoing issue causing customers to receive error notifications when performing service management operations - such as create, delete, update, scaling, start, stop - for Virtual Machines (VMs) across multiple regions. These issues are also causing impact to services with dependencies on these service management operations - including Azure Arc Enabled Servers, Azure Batch, Azure DevOps, Azure Load Testing, and GitHub. For details on the latter, please see https://www.githubstatus.com.

Current status: We have determined that these issues were caused by a recent configuration change that affected public access to certain Microsoft‑managed storage accounts, used to host extension packages. We are actively working on mitigation, including updating configuration to restore relevant access permissions. We have applied this update in one region so far, and are assessing the extent to which this mitigates customer issues. Our next update will be provided by 22:30 UTC, approximately 60 minutes from now.

https://azure.status.microsoft/en-us/status

llama052··on GitHub experience various partial-outages/degradations
Looks like Azure as a platform just killed the ability for VM scale operations, due to a change on a storage account ACL that hosted VM extensions. Wow... We noticed when github actions went down, then our self hosted runners because we can't scale anymore.

Information

Active - Virtual Machines and dependent services - Service management issues in multiple regions

Impact statement: As early as 19:46 UTC on 2 February 2026, we are aware of an ongoing issue causing customers to receive error notifications when performing service management operations - such as create, delete, update, scaling, start, stop - for Virtual Machines (VMs) across multiple regions. These issues are also causing impact to services with dependencies on these service management operations - including Azure Arc Enabled Servers, Azure Batch, Azure DevOps, Azure Load Testing, and GitHub. For details on the latter, please see https://www.githubstatus.com.

Current status: We have determined that these issues were caused by a recent configuration change that affected public access to certain Microsoft‑managed storage accounts, used to host extension packages. We are actively working on mitigation, including updating configuration to restore relevant access permissions. We have applied this update in one region so far, and are assessing the extent to which this mitigates customer issues. Our next update will be provided by 22:30 UTC, approximately 60 minutes from now.

https://azure.status.microsoft/en-us/status

llama052··on Pricing Changes for GitHub Actions
Oh absolutely, but honestly the self hosted runner setups that I'm familiar with are just waiting for a call. As far as I can tell GH side just routes.
llama052··on Pricing Changes for GitHub Actions
Our org is showing around 200-300$/mo in added fees and we are exclusively self hosting in our own on premise cluster. Kind of wild we have to pay to use our own compute.
llama052··on Pricing Changes for GitHub Actions
I guess this is on brand for Microsoft. Just lame to go through the trouble to self host runners and still get tacked on with fees after the fact.

Hard for me to feel like our industry is innovating and not just gouging with the rest in the battle for enshittification.

I will intentionally start exploring other options even if the cost isn't high, because I don't want to support this type of thing.

llama052··on Upcoming Changes to Let's Encrypt Certificates
We use locally generated certs for Mtls with different lifetimes. Relying on public CAs for chains of trust like that makes me nervous, especially if something gets revoked.
llama052··on [dead]
I’d argue that the big 3 cloud providers have more outages than this, only cloudflare actually lets you know.
llama052··on Average DRAM price in USD over last 18 months
It's more advantageous long term for them to be oblivious to it. Ultimately gives them what they want which is reduced supply and increased pricing for them.
llama052··on Average DRAM price in USD over last 18 months
I think I explained that poorly. Basically artificially reducing supply so that these manufactures can get more for less. They've been caught doing it in the past before between each other, so why not use OpenAI as a bridge for that.
llama052··on Average DRAM price in USD over last 18 months
The cynic in me thinks this would be a convenient way for these memory producers to manufacture demand, while also making OpenAI look good on paper. It’s not like they haven’t been caught price fixing in the past. Win win for these companies and a loss for everyone else.
llama052··on Tell HN: Azure outage
Please migrate off of front door. It's been a failure mode since it came out historically. Anything else is better at this point
llama052··on Tell HN: Azure outage
I got a service health alert an hour after it started, saying the portal was having issues. Pretty useless and misleading.
llama052··on Tell HN: Azure outage
Always fun when you can't trust the main status page but have to go to some opinionated social medial website to see the actual problem.
llama052··on Tell HN: Azure outage
Just another day with microsoft. Honestly pretty tiring as something is always generally broken.
llama052··on Show HN: K8s Cleaner – Roomba for Kubernetes
Agreed, this feels like the kubernetes descheduler as a service or something. Wild.
llama052··on Reclaim the Stack
So you had auto update enabled on your cluster and didn’t keep your apiversions up to date?

Sounds like user error.

llama052··on Delta CEO calls Microsoft 'the most fragile platform' while praising Apple
I think it's mostly driven by a few things..

A. Doing security is expensive and viewed as a cost burden at a lot of non-technical focused companies. Lots of businesses hedge their bets hoping that a security incident won't be as expensive or detrimental as having a great security posture. Sadly often times they aren't wrong either.

B. Security compliance standards are dated and opinionated, requiring rigid solutions to complex ever changing security threats.

Both of those can drive the narrative of pushing for tooling that offers the least amount of resistance to implement and be able to claim "secure".

Additionally IT and Operations teams are constantly getting more duties and can be some of the first teams to get rightsized and viewed as "cost centers" in some companies. I've seen teams reduced 50-80% over the years with expectations higher and security compliance becoming the last on the list and then gets the least amount of energy and attention.

llama052··on Delta CEO calls Microsoft 'the most fragile platform' while praising Apple
Security compliance requires all sorts of "invasive" tooling to ensure your client workstations and servers are "safe". Sadly it's mostly a checkmark and often times has dated and arbitrary requirements. As far as I know CrowdStrike was one of the easier ones to setup albeit expensive.
llama052··on Tau: Open-source PaaS – A self-hosted Vercel / Netlify / Cloudflare alternative
I assume https://www.talos.dev/

Basically a small OS that will prop itself up and allow you to create/adopt into a Kubernetes cluster. Seems to work well from my experience and pretty easy to get set up on.

llama052··on Tau: Open-source PaaS – A self-hosted Vercel / Netlify / Cloudflare alternative
100% this.. There's also exciting projects like Talos, Rancher, and the like for self-hosting Kubernetes that makes it entirely more manageable.

So much saturation in this space of people trying to create one off solutions, which on some level I admire. However the further off the main path you go the more you lock yourself into problems you can't troubleshoot or edge cases that aren't supported.

Abstraction these days is alluring, and it's cool! However you want something well known, well supported, (from multiple companies ideally) and documented. The hate for understanding kubernetes is just hate for having to understand layers of orchestration, or worse the layers behind the application.

If it's too complicated then you might not need it. Any platform you use will have those same layers, it just depends on how much is assumed or exposed to you. If you don't want to see any dials or options then use a managed solution, not a roll your own platform tool. That's of course assuming a few virtual machines managed by hand doesn't satisfy your needs, but if that's the case you don't need a platform solution (and hopefully it's not production).

llama052··on A Eulogy for DevOps
Nothing specific I've read that helped us with this, our goal was finding the easiest path to entry to get the tooling we desire in our platform.

I've debated writing it up and posting it somewhere, maybe I should. There's so many ways of doing things now that it's quite overwhelming sometimes.

llama052··on A Eulogy for DevOps
So there should be magic on every layer except for the application?

That will never happen, the only thing you can do is pay someone like heroku to take care of that for you. Or if your project is small and plain enough you run “serverless” which is just routing to another platform team, as I’m sure you’re familiar.

It’s complicated because there’s a lot that goes on, kubernetes or not.

← PreviousPage 2 of 7Next →