Ask HN: What could developers do to help the Ops Team? | Hacker News Reader

Ask HN: What could developers do to help the Ops Team?

10 pointsvictorcase22 comments

I'm researching about DevOps and what the dev team can do to improve the Ops Team efficiency/productivity? I already have some feedbacks of local teams, however I would like to listen what is your pain.

View on HN

I work mostly with deployments and updates to our environments, and the most helpful thing my developers can do for me is provide validation steps to confirm the state after each change.

We've caught issues ranging from expired passwords, missing files / files with incorrect permissions / write failures, sync issues and other obscure gotchas.

Catching a failed step during deployment can sometimes prevent a huge rollback effort and save a lot of time.

What about containerzired deployment? Isn't Docker the silver bullet? Or at least will it allow to mitigate your issues easily?

Docker alone isn't remotely close to comprehensive test driven infrastructure.

Appreciate why conservative technology choices are made when possible (pick boring tech).

Don't rush new features or code into production. Don't deploy anything after 5pm on a Friday.

Documentation, documentation, documentation. If there's a piece of knowledge for your app that's only in your head, you have failed.

What you think about sandbox applications (like docker) over a PaaS can it mitigate a heterogeneous scenario?

Containers almost always for local dev environments. I don't recommend containers in production unless you're going all in on a container scheduler (like Kubernetes). Without running multiple containers on the same virtual machine (and and orchestrator performing the work for you), you're adding an additional layer of abstraction with no benefit.

Thanks for the reply, but can you explain (or provide some reference) about why not to use containers-only in production?

I know that I will add a layer of abstraction, however from a Dev perspective I have some benefit like a byte-to-byte compatible test/prod env that I haven't to worry about dependencies per example.

I always feel being scared of deploying on a Friday is a massive red flag that deployment proccess, does not have good automatic testing, doesn't use canaries and does not have seemless roll backs.

Nowadays we still having the scary of deploying on Fridays. What are we missing to lose this fear? Isn't DevOps mature enough? How big enterprises like Google/Facebook/Amazon.. are handling it?

We deploy on Fridays because our app is used only from Monday-Friday and if anything goes wrong we have the whole weekend to fix it, but we've never actually had to do that.

The Deployment is a tension point of Software Development Life Cycle[1]. The more we deploy less scared we are.

[1].:http://dl.acm.org/citation.cfm?id=2593813

Caring about and fixing inefficient wasteful tests. Stops CI build queues clogging up.

Also, improving production performance/scalability.

And making apps simple to deploy. That's about it for me :)

What means simple? Also, do you talking about to stop CI/CD flow? Why not to provide a way to keep that flow on production?

> What means simple?

Just things like not requiring complex set up steps and wiring up many different services with complicated configurations etc. Keeping things stateless (as much as possible) etc

> Also, do you talking about to stop CI/CD flow? Why not to provide a way to keep that flow on production?

Not sure what you mean, but slow tests can be a major problem. Most of the time these end up being due to no thought going in to not repeating the same setup steps zillions of times. As new tests are constantly added, it makes it harder and harder to keep builds/ci fast.

This is the problem that we need to solve;

"How develop/maintain a complex software and at same time keep it simple to set up without a huge bottleneck with Ops team"

If state(ful) is a problem, How about KILL Stateful? Maybe we need to (re)think about how we are handle with microservices? If true, that's is an good hypothesis to validate in a dev perspective.

Developers with domain knowledge should often be the first line of response for pages, etc. Noise in logs should be minimized to aid troubleshooting.

Understand why certain security procedures are put in place even though they do create some friction.

Clear and specific requirements for environments you need.

Good tests, with good code coverage.

Work with us when we suggest to you that we deploy to a non-production environment first.

Work with us to automate as much as we possibly can.

Thanks for the tips; However.. We know that specific requirements are unreal for startups-like and small enterprises right? Then.. Do you have some suggestions about how techniques we can apply to handle with dynamic requirements? Maybe the real question is : What software architectures/patterns are more friendly to help Ops with it.

Learn OPS in order to relieve their workload.

To Learn requires time. Also, developers and Ops have a lot of work to do. Why should I spent time to learn a lot of things about ops. I want to code

Can you list at least five principles that a DEV must Learn?

I was half-kidding with my reply, because a company should realize when there is a bottleneck and address it as long as the resources are available.

But, this isn't always the case, so here's something a dev can do:

- Know basic server administration. Nothing too crazy:

  - User roles and permissions setup.

  - Database access and setup.

  - Webserver access and setup.

- Be the master of your dev environment. If the organization uses virtualization for everything, learn how to manage those virtual machines, or at least, know where to find the config files and where to find the documentation in-case you need to change the config files. Read through the config files so you at least feel confident navigating them.

- Know what happens when code is deployed. Where does it go? Which servers? How to log into those servers to debug issues? Where are the logs? Ask what you'd need to know if you were suddenly put in-charge of the the project as the sole-developer-webmaster type person.

I think if a developer is able to do the above, then their devops team will think more of them and maybe be more willing to help when things aren't working out.

Reply on news.ycombinator.com