Hacker News Reader

Comment by bigstrat2003 | Hacker News Reader

bigstrat2003

I mean... one certainly could do this. And then as soon as anything breaks with that container, the person trying to fix it will curse the day you were born. People already go way too far with stripping container images, and it's awful for troubleshooting. The last thing we need as an industry is taking that silly trend even bother.

View on HN

If you're going to use minimal container images you definitely need to know how to inject tools into the container.

Yeah and it's a true pain. Double that if your container security policy requires running in fully rootless mode (i.e. not just the usual CAP_ALL drop but also container uid != 0), quadruple if you're in a Kubernetes environment, and even more if you can't access the nodes themselves directly (AWS EKS).

I'd say you need to know how to properly instrument your containers so that you'll never need to get inside. If your team is on the level of "we need tools inside the container for troubleshooting", then don't use minimized containers like these.

Developers having shell access to production container is not the best practice, to say the least. For debugging locally, it's very easy to nsenter your container.

I think the point is that you should have a plan (which may or may not require debug tooling in the image) and you should include what you need for that plan and nothing more.

Finding something useful that the distro left lying around but that wasn't part of the plan might feel like a win here and there, it's just as likely to be a win for your adversary as it is for your friend.

It's about security and reducing the probability of getting CVEs for your running containers. Less software you bundle there, less security issues you will have. And this in turn means less work on maintenance as you don't need to update, test and deploy new versions of images so often.

We use Google's distroless containers for production, but we also have alternate Dockerfiles available in our CI/CD for building a more traditional container that can be logged into if there are application issues to debug.

It's not an exact copy of prod, but so far (a few years, and thousands of containers) it hasn't really been an issue as far as I've heard. We don't do anything fancy though. Node, Go, Java and Python apps..

When the cow gets sick, you shoot it.

You still want to know why the cow got sick, in order to prevent all other cows in the stable from falling sick as well.

And ideally, you want to keep the cow alive, at least if you fed it artificially.

You're not looking at this from a DevOps perspective.

In real-world cloud environments, logging into the container and expecting to have a complete Unix environment to poke around in just isn't an option. You might not even have the required permissions to do so, as a matter of corporate policy. You need to build observability directly into the application -- or you haven't built a complete cloud application.

> You need to build observability directly into the application

This is not objectively better than external observability.

If the "DevOps perspective" requires it, then it's a pointlessly inflexible perspective.

> just isn't an option

You don't get to blame the system when you made the system.

The option is there.

That's the key thing why DevOps is frankly a pile of bullshit and why so many companies ridiculously overpay on cloud infrastructure.

When you got clueless morons who were in a 6 week nodejs bootcamp now "developing" containers, with no one even having the permissions to troubleshoot crashes - that may very well originate from someone trying to get remote code execution - it is absolutely no surprise that hacks, ransomware, infrastructure so brittle that it only doesn't fail because EBS / EKS keeps restarting the pile of dung every minute and infrastructure pieced together from 40 AWS services where one classic EC2 VM would be sufficient are plentiful.

Reply on news.ycombinator.com