Hacker News Reader

There may be a rare exception here or there where network effects or some other unique market need require a startup to treat their infrastructure as cattle, but otherwise it's absolutely true. Running `git init --bare` on a server, setting up some remote origins, slapping some bash scripts into the git hooks folder, some .service files for systemd, and then running `git push origin dbhost && git push origin apihost && git push origin ...` is a hell of a lot simpler than even getting a toy version of kubernetes up and running. For the developer with average linux skills, it's probably even easier than figuring out which cloud orchestration monstrosity to choose. Throw in Github actions and baby, you've got a stew going!

Maybe I'm misunderstanding the whole "pets vs cattle" thing but you don't need to lose repeatability or give on process altogether; it's a false dichotomy. Just don't dive headfirst into the massive complexity that is "scalable" devops - you can still use terraform or docker compose or whatever to deploy your application and provide repeatable development/staging/test infrastructure.

> is a hell of a lot simpler than even getting a toy version of kubernetes up and running.

Getting it off the ground is one thing, keeping things reproducible so you can stand it back up reliably if the instance goes down is another thing. You need to know what are the relevant changes that people have made to the instance over its lifetime in order to reproduce them on the new machine. Maybe this is not what people mean by "pets vs cattle", but they seem pretty closely related if nothing else. I guess the thing I care about is "reproducibility" and for all of its complexity, Kubernetes makes the happy path pretty clear in this regard whereas with EC2 you need Ansible and a whole bunch of playbooks to reproduce things like SSH, logging, monitoring, process management, etc that Kubernetes or ECS or whatever give you out of the box. And if you find that you need to scale beyond one or two instances (not everything is a CRUD webapp, after all) especially autoscaling, then it seems like the EC2 story becomes even more complicated (you now need something to automatically invoke Ansible with the right playbooks for each node). I'm probably biased by my experience (aren't we all?) but it seems like Kubernetes or ECS/Fargate are simpler in these regards.

> And if you find that you need to scale beyond one or two instances (not everything is a CRUD webapp, after all) especially autoscaling, then...

Have you ever been working a solo project that needed autoscaling? If so what was it doing that was so resource intensive per $ earned that it was still a solo project?

Picture an analytics application--something that has to crunch a bunch of data per request, and your users are mostly visiting from 9-5. Depending on the size of the user's data source, they could easily use a whole CPU and a lot of memory for each request and requests can take up to 60s or maybe even multiple minutes. You need to run more than a handful of machines, so you can either overprovision or autoscale, but in either case you need some amount of reproducibility to wrangle all of those machines and you'll probably want to plan for them to come and go because you probably don't want to pay for a bunch of high-end machines sitting idle every night. Optimization is nontrivial so you'll get around to it eventually and maybe even hire, but hiring your first employee is a lot harder than hiring your 101st employee, so you're not going to do it right now but you still need a solution to your problem...

Did you run an app like this? If so, which one?

I very nearly launched an event analytics app at the beginning of this year but ended up just rolling one for my own internal use.

I did in a past life. I'd prefer not to say which one because this is an anonymous account and I don't want to make it too easy to associate to my real world identity.

Understandable! Thanks confirming that it was a first-hand experience rather than a hypothetical.

> Getting it off the ground is one thing, keeping things reproducible so you can stand it back up reliably if the instance goes down is another thing.

Are we talking about pet servers or kubernetes here? Jokes aside, I think you're underselling how complex Kubernetes is and overselling how complex logging, monitoring, process management, et al are. You don't get any of those things with kubernetes "out of the box" unless you've got an ops team or someone's got a year plus experience running one cloud provider's version. It seems really easy cause we've got one click clusters and a bunch of great UIs but all the time people spend chasing down problems and wrangling the extra complexity of k8s in production gets papered over because it's the hot new thing.

The bash history for the setup above fits on a single page and since the advent of Nix and now Terraform, repeatability hasn't been a problem. The harder part is discipline among coworkers but Linux user/group permissions go a long way towards making it readonly for debugging purposes while still giving fine grained control over servers.

> Jokes aside, I think you're underselling how complex Kubernetes is and overselling how complex logging, monitoring, process management, et al are. You don't get any of those things with kubernetes "out of the box" unless you've got an ops team or someone's got a year plus experience running one cloud provider's version.

I'm pretty sure cloud providers give you logging and monitoring out of the box (as well as a bunch of other stuff) and Kubernetes itself is a process manager with its own "SSH" (kubectl exec) and so on. I'm not wed to Kubernetes either--you could use ECS/Fargate or whatever. But to do something comparable in an EC2 environment requires a fair amount of experience just to get to get the basic things that Kubernetes (or ECS) gives you out of the box. Of course, if you're well-versed in VMs already then I don't think you'll get much value out of a container solution for simple workloads, but if you're not familiar then I think container solutions are easier. But again, I might be biased by my experience.

I think it entirely depends on the path you're taking. For example, I'd disagree with the quoted section you made for a different reason. I'd almost always suggest early stage startups to use something like Heroku (if it works for them), which gives them the ability to pay a small premium in order to never have to spend too much time dealing w/ devops stuff. Pre product-market-fit, that's wasted time/effort unless specific devops needs are essential to getting there.

> Is this true? At my last company we wasted a bunch of time every week dealing with the accumulation of ad-hoc changes in different environments creating different behavior.

That is an issue a disciplined one-man operation won't have since (generally) they have a Dev and a Prod environment, the changes would be made in Dev first then duplicated in Prod.

If you have 2+ developers, you have environment drift like you experienced because (generally) Dev is the local environment on the laptop or otherwise a distinct environment for each developer. So now you have 3+ environments you need to sync and then discipline starts to fall apart as you have to communicate every change.

What happens to the disciplined single dev when their EC2 instance falls over in either environment? How do you get things back in sync without code to automatically stand things up in the right state? And if you're going through the work of writing/maintaining that code, are you saving anything over using Kubernetes (consider out of the box, Kubernetes gives you logging, ssh, process management, service discovery, etc)?

Not everyone uses EC2 which is prone to fall over.

And the disciplined single dev documents their changes. That is why it works in a 1 man shop but not at scale, there is always someone who is unreliable once the group gets large enough.

My take would be that once you need another environment than staging+prod, that's the time to move to at the very least move to containers, and probably think very hard about either a hosted Kubernetes or something like Fargate.

When you say "we" it sort of implies this was not a one-man operation.

You definitely want "one click" deployments from the git go. Then it's just a small tiny step from that to your servers being cattle.