Hacker News Reader

Most enterprise developers aren’t allowed to know the answers to your questions as none of them fits into a feature rooted in a quarterly OKR. For most enterprises developers it takes a couple of weeks to create a git repo because it requires a service ticket to Devops.

Whenever speaking with pure developers, inevitably I will hear this sentence:

"I just want to...."

It's terribly rude, but I stop paying attention, because inevitably the rest of the sentence can be translated to:

"... ignore my responsibilities, such as security, observability, availability, privacy, security, recoverability, documentation, scalability, and long term supportability.

That's an interesting way to frame it.

I would say the context of such a sentence is usually something along the lines of:

"Because X wasn't designed with <problem> security, observability, availability, privacy, security, recoverability, documentation, scalability, and long term supportability </problem> in mind, to just do Y it is now also my responsibility to do <problem>"

I've had these conversations with developers facing the proverbial blank canvas of a new Visual Studio project they had created mere days ago.

Excuses are just that: excuses.

> I worked with a Microservices Architect (and developer) once, and when I asked how he intended for failures to cascade properly between services; how he would maintain cohesive state, especially if things went wrong; and how he would diagnose problems between them, he confidently told me: "That is for devops to work out."

JFC, that's an incompetent.

I know a lot of people will consider that too strongly said, but I don't think so. Any jackass can succcessfully build for the happy path, it's precisely where the happy path isn't that real skill comes into play.

What answer did you want to hear? Mine would be "Goes to dead-letter queue to be reprocessed OR fails, logs to central logs, team gets an alert." Haven't seen "distributed logging" magic work in practice. I have co-workers who spent years on tracing request ids through each layer. It got scrapped after a couple of months because no one used it.

> I have co-workers who spent years on tracing request ids through each layer. It got scrapped after a couple of months because no one used it.

"Hey bob, it seems the money we sent through the system disappeared. Can we trace it?"

"Sorry no, we got rid of the reconciliation id we use across our systems."

That sounds cute to you, but it's a huge waste of time for most applications. Pulling out the banking example is a cliche at this point.

If it's a move money issue - that's solved - transaction to credit/debit. Reject and log if failed. Maybe with a retry. Maybe from a queue.

If it's financial documents moving, they go through a serious of queues and get dead-lettered and handled appropriately.

The kicker? All of these have a customer / transaction ID and don't need some special request id "traceable throughout the system" - it's traceable as is, search for it by customer / transaction ID.

This even applies if you are using K8S for no reason. The only company I know that's justified in using K8S is an AWS/GCP/Azure competitor in the ML space needing to spin up VMs instantly at scale.

k8s saves a TON of time if you need to run on more than 1 machine and you use a managed solution (by AWS/GCP/Azure) and only use it as a glorified docker container orchestrator. Ingress solves how to route HTTP traffic coming into your domain. PersistentVolume solves how and where to store data. LivenessCheck solves how to restart your crapware when it freezes. Just don't go crazy with it unless you actually need the fancy stuff.

"k8s saves a TON of time if you need to run on more than 1 machine and you use a managed solution (by AWS/GCP/Azure) and only use it as a glorified docker container orchestrator."

How? Just use AppService / EC2 for managed machines. Scales just as well.

"Ingress solves how to route HTTP traffic coming into your domain."

This is actually helpful and "easier" out of the box than putting nginx in front of statically hosted sites, so I will reluctantly give you this one.

"PersistentVolume solves how and where to store data."

You need a DB most of the time. Managed DBs are easier.

"LivenessCheck solves how to restart your crapware when it freezes. Just don't go crazy with it unless you actually need the fancy stuff."

Our crapware doesn't freeze. Needs a custom Health Check endpoint. Can just as easily restart the instance when that check fails.

K8S is a ton of extra overhead for no gain. We have this debate at work all the time. Great resume builder though.

> How? Just use AppService / EC2 for managed machines. Scales just as well.

It's not about scaling. It's about being able to make a docker image, run that locally on your developer laptop (even if it's a mac) and running the exact same thing in production. Sure you can use docker on top of EC2 but why would you? Just use the ready to go solution you can just buy (managed k8s) and avoid the lock-in. Again, it's critical that you use managed k8s. Don't roll it yourself on top of EC2.

> You need a DB most of the time. Managed DBs are easier.

Sure. Using k8s for your app isn't stopping you from doing that if you want. We deploy 1 database server per testing environment, which kubernetes makes easy to do. For production we use a managed database, but they're too expensive for testing environments.

> Our crapware doesn't freeze. Needs a custom Health Check endpoint. Can just as easily restart the instance when that check fails.

Mine doesn't either. But I have to run plenty of software that isn't mine.

> K8S is a ton of extra overhead for no gain

If you used managed k8s you're just trading server management for kubernetes resource management. I've migrated our system from Ansible + Cloud VMs to kubernetes several years ago, after running it for years before, and things have become significantly easier to maintain and our costs are roughly the same as before. Less vendor lock-in, too. It would take me an about an afternoon to switch between any of the major managed kubernetes offerings, and some of our clients have expressed preferences for a specific cloud.

Hey, thanks for the reply. I am being slow because it looks like if comments are deeply nested, the "reply" cooldown gets longer, so I am just replying now.

We use .NET/React for the most part, so I haven't had an issue with Mac/PC - we have multiple teams and, for example, a team helping is all on Macs - they can run stuff fine from bash. Azure Data Studio let's them access Postgres (and would let them access SQL Server) if needed. React only has issues with file pathing sometimes, so most of the time it's fine. I find that devs also get lost in setting up Docket/K8S locally, especially when people roll scripts to set up the whole environment in one command.

Still, I am genuinely arguing to help me fill the gaps where I might be incorrect, so I really appreciate your replies.

On the flip side I've seen dev teams grind to a halt because of how much difficulty they had tracing a request end to end in their system.

Granted, they had entirely too many separate web services, and that was really the root of their problem, but being able to trace a request all the way through your system isn't useful until it is (meaning, when shit hits the fan).