Hacker News Reader

Comment by brozaman | Hacker News Reader

Becuase your solution is specific to your specific software.

Then you have lads who use python who can use virtualenv, and then the ruby ones who use rbenv and then you have the CPP developers which are usually fine but on Tuesday Joe messed up the linker settings and all of a sudden we're not statically linked, thanks Joe.

And then our Java team forgot to tell the sysadmins that they upgraded the JVM version.

The thing with docker is that you just get a complete package with everything delivered directly by the developer and it's very hard to mess up.

Also docker can run applications that aren't shipped by people in your company and also does more stuff (like more sandoboxing and it also helps you with the resource management).

View on HN

> The thing with docker is that you just get a complete package with everything delivered directly by the developer and it's very hard to mess up.

The downside is: you are also, at the same time, excluding yourself from standard patch management. Whereas in ye olde times for a situation like Heartbleed or log4shell the 24/7 on-call sysop team would run a simple "apt-get upgrade openssl" and restart all affected daemons, under modern "devops"/docker-centric models you need to wait for the developers or, worse, the container image vendor to deliver updated versions.

Not really; if you have a few servers vs. a few containers the patch is the same with the difference that the server one is probably "live" patching on a running system and the docker one is patching it in the docker file and re-deploying which makes it an atomic change.

It also really makes a difference if you are combining some tasks and ownership in a DevOps concept where it's not just developers handing off some binary and telling the operations team "good luck eh". That classic setup mostly brings finger-pointing problems and not really much of a solution.

What a patch management concept would look like is much a more modern version of the server thing: you have some system that allows you to query all the servers "what is your current version of thing X" and if you then want to change that you build an updated machine image and deploy that to all those servers. Cattle, not pets.

The problem is, most ops-focused people don't have much knowledge of the boatload of different application-side tooling, and most devs (especially those out of "coding bootcamps" or practical academia) don't have a single ounce of knowledge about OS management or even how basic UNIXoid fundamentals work.

DevOps has always been more about companies attempting to "save" money by shoe-horning together roles that require completely different skill sets.

> and most devs (especially those out of "coding bootcamps" or practical academia) don't have a single ounce of knowledge about OS management or even how basic UNIXoid fundamentals work.

I have to vehemently disagree with this statement. I had 10 years in IT before I did my bootcamp and quickly found myself in deployment engineer roles by accident. Most of my bootcamp cohort ended up at smaller companies wearing many hats and learning quickly where a good half of them are in deployment type roles by necessity.

After an additional 10 years in this industry, I'm finding invariably that it's the FAANGs and companies hiring fresh grads out of the best schools where engineers aren't learning anything about the underlying systems. It's the feature factories where people put their headphones on, the heads down and code like their life depends on it.

They do things like write microservices that implement basic functionality of opensource loadbalancers, because they've never heard of nginx or haproxy before, don't know how it solves their specific problem and wouldn't know how to configure it. When all you do is write code, every problem looks like it should be solved by writing code.

That hasn't been my experience in the last decade or so, and new people or people with different specialisations are indeed unlikely to be good at everything on the first day.

Before the overlap between development and operations there were multiple teams involved in every step, everything was slow and expensive and you'd be lucky to deploy once a week. After making sure that where overlap exists the people that have the skills to work in that area can do what they need to do (i.e. build, verify, release and instrument the systems they are responsible for) we get every team releasing every system multiple times per day in small increments. This is both faster, easier for everyone, and worth the ROI on people and tech far more than trying to invent borders and fiefdoms between hyper-specialised teams.

Is depending the container image vendor actually worse? I was under the impression that they were about as "on it" as the distro maintainers with updates. And if you wanted full control then you could just have your sysadmins manage their own base images. Then upgrading system dependencies becomes as simple as doing a regular deploy.

Sometimes it is. Depends on the vendor, of course, but I've seen them take many hours/days after a security fix is released.

I feel like I get this a lot with the NPM cli tool ... No matter how often I update my dockerfiles to the latest version, I'll always end up with a big banner warning message in my build output telling me I'm outdated 'cos it seems NPM updates (and publishes) constantly.

I don't use npm all that often, but I don't think I've ever run "npm install" in a project and not gotten any warnings about outdated dependencies and security vulnerabilities in them.

it gives a bit of a 'broken window' effect, 'cos while it still irks me now, I'll probably start turning a blind eye to all these warnings only to miss something really critical later on.

There are container management systems that can inform you of container images requiring patching. You can specify the policy for how to deal with these container images: do you quarantine them, do you stop running that application, or whatever. TBH, you're presenting a happy path scenario that often fails to materialize. Sure, "apt-get upgrade openssl" and restart all affected daemons, keeping your fingers crossed everything will still work. It often doesn't, and now your system is borked. That's not a good way to run an operation.

>And then our Java team forgot to tell the sysadmins that they upgraded the JVM version.

That's just a sing for bad communication/practices/documentation and processes, and why don't the dev's test the application on the dev/test-environment?

BTW: Why can the JAVA-team even install something on the dev/test/prod-env? That's completely against the rules.

Because in reality, with large teams, communication is expensive.

Containerisation allows reproducibility, in a self-documenting manner. Delivery can be automated, depended upon to run, in a totally system configuration agnostic manner. It allows projects to deliver their software in a way that it can be ran, instantly, with a single command (image download not withstanding).

I've worked heavily in service software engineering for over a decade now, and with docker since 2013, and it has changed everything. There will be some people who don't fully understand it, but perhaps they don't have the problems that it solves and thus don't need it.

Alternatively, they /could/ just be suffering without realising there is a solution.

You can think of docker files and other manifests as a highly structured and more direct-to-machine form of communication. Instead of a developer informing other developers to ensure that the dev machines and servers are upgraded to the right version of the JVM via a confluence page or slack message, that developer has left machine-readable instructions in a consistent format in a readily accessible place.

This is a good way to fix those problems with communication/practices/documentation and processes.

>This is a good way to fix those problems with communication/practices/documentation and processes.

No it's not because the dev who made the database-container forgot to inform you that you need the new jdbc.jar etc...it's the same just with one more layer.

Yes. However, with more and more teams, and more and more languages, this communication, and especially planning and executing these upgrades and changes while providing rollbacks for all languages becomes harder and harder. It's very possible to do this, but it takes a lot of effort.

Containerization largely eliminates this need for communication. There are new problems - security of running containers, image sizes, building images safely and so on, but this problem disappears.

Well yes you are absolutely right, the first comment who sees the down and upsides clearly.

> why don't the dev's test the application on the dev/test-environment?

They did, it failed, then the JVM was upgraded for dev/test (but not yet for prod as that was still running the prev-JVM version of the code).

>They did, it failed, then the JVM was upgraded for dev/test

See first error, dev/test has to be a exact mirror of production, otherwise it makes no sense. And if not even the admins from dev/test/production can communicate with each other...you really have deep rooted problem.

Reply on news.ycombinator.com