Hacker News Reader

Comment by wizofaus | Hacker News Reader

I just quit a job partly because we lost our key DevOps guy and no serious effort was made to replace them. As a result I ended up wasting huge amounts of my time dealing with operations-level stuff that made it impossible to focus on the key parts of my role (feature development etc.). I subsequently turned down a job offer from elsewhere that explained their policy was not to have dedicated DevOps resources for their SaaS platform (devs themselves being responsible for all deployment and system maintenance), and would do so again. Good DevOps people are worth their weight in gold, and at least in many verticals (e.g. those involving payments ) it's virtually mandated that there is a separation of responsibilities between those writing the code and those responsible for delivering the product to customers. I can't see the need for dedicated DevOps resources going away any time soon.

View on HN

> I can't see the need for dedicated DevOps resources going away any time soon.

But is what you're describing just.... "ops" without any of the "dev"? I'm not saying that there is not a need for dedicated infrastructure and operations teams at a certain size (and in some industries), but that's not an excuse for devs to feel like they can chuck a new feature over the wall and say "Well, I'm done my job. Please run it and make sure it doesn't break".

No silly, that's what the QA team are for! Anyway, our DevOps guy spent probably most of his time "developing" - just not application-level features.

Agreed. It also goes back to the sys admin problem which devops optimised in that the Dev team owns there code in production.

I've seen success with a dedicated devops member on a team but having a dedicated devops team just introduces delays and latency when fixing pipelines or releasing. The very same problem we had with sys admins.

At a certain point, hiring dedicated Devops frees up x number of developers to continue to develop features depending on the amount of time each developer is spending on performing those devops duties. It’s just another area management can split up job roles to capture more value and allow deeper specialization among professionals.

Sure, but if you're hiring people in a different role that does all of the ops work for developers, this isn't "devops". This is just "ops".

Sounds like a terminology issue then, I consider it "devops" because they're mostly writing code that goes into our git repo etc. etc., they still have to do PRs and code reviews etc. But the code is to handle deployments, not to implement features.

Yes: the terminology problem is the is you are using the terminology wrong.

DevOps is about cross-functional teams, and has been co-opted by vendors to sell products. And delivery of software is the ultimate feature: without it, nothing of value can be produced.

Then everybody else I've worked with over the last 6 or 7 years is also using it wrongly.

Quite likely.

Well then we're just getting into debates about what determines the "correct" meanings of word. I read the original article and most of the comments here using my understanding of the term, and it made sense that way...

(not to beat a dead horse, but I just finished some interviews I was involved in as an advisor and the term DevOps was used a lot in all of them. And in every single case the term was used exactly the way I've understood to mean - i.e., DevOps engineers are those who primarily look after the CI/CD pipeline, generally don't write application-level code or develop features, but do very much spend the majority of their time developing scripts and tools to enable CI/CD).

I thougt the "dev" part was because the "ops" were using code to build the infrastructure.

In my experience that leads to meh code if the "devops" comes from a former sysadmin, or meh infrastructure if s/he comes from the dev side.

I keep hearing that "we are all devops" from my PM. The real kicker is that there is a dedicated DevOps team in my organization, they "just have too much to do already".

“Dedicated devops” is not devops, never has been. Devops is a culture where, super simplified, you run what you built. “Dedicated devops” is just ops

I get some people use the term that way, but in our case, our guy really did spend ~50-60% of his time developing IaC and other deployment scripts/tools (in various languages), and the rest handling the operational side of things. He occasionally touched the application code as needed (renaming configuration variables etc.), but he didn't do feature-level development, nor was he interested in doing so.

This was also my experience.

I used to work in an org with about 200 engineers, supported by a DevOps team of ~6 devops engineers.

They spent (very roughly) half of their time doing operations, and half of their time writing software to make devops easier.

Every other month they’d announce some new tool for us to run automated tests or speed up builds. It was awesome, and made our developers more likely to do their own portion of the ops work.

> They spent (very roughly) half of their time doing operations, and half of their time writing software to make devops easier.

This is what happens in my (non-tech-industry, but heavily invested in tech stacks) employer. A previous manager decided we were no longer System Administrators but now DevOps Engineers. But none of us develops code for our features and none of us wants to develop code for features. We want to build and maintain the infrastructure on which the code runs. Some of it has to be run in-house because we deal with medical information and all that entails.

Our current manager describes not as Developer Operations but Developing Operations. We develop the infrastructure and tooling that our code writers need and do everything we can to get the moving parts out of their way. If the code written by the code writers fails, it is their responsibility to deal with it but if the infrastructure underpinning that code fails, my team has failed.

Ime this unravels quickly as the team and stack complexity grows and product engineering starts shipping software that’s very difficult/expensive to operate. The whole idea behind devops was to avoid this death spiral by giving engineers ownership and responsibility to run their own software. The problem was (still is) that off-the-shelf tooling is just not there and requires very skilled/experience devs to use it effectively. Hence dedicated teams got created and the whole thing died on the vine. Devops as a movement simply doesn’t scale.

I don't see it as massively different to specialisation between front and back-end developers though - we're all devs, we're all comfortable with reading/writing code, but whereas my specialty and focus is on application-level (typically backend) code, the DevOps guys are specialised in writing IaC scripts etc. Yes, in principle, any dev could do it all, but there becomes a point that the mental load from juggling too many technologies outweighs your ability to be productive. And some types of development just take different mindsets - I've done my share of front-end/UI-type development in the past, and can easily pick it up again if needed, but I can't say I find it super satisfying, and it's nearly always going to be produce a better result for everyone to hand it off to someone who does. Likewise for CI/CD scripting. And it also happens that if your primary focus is on the scripts and tools necessary to get software deployed to a particular hosting environment, you're likely to have the skills necessary to take care of the day-to-day sysadmin side of things for that environment (indeed, most "manual" changes to the environment are only done as a last resort, and would be wiped by the next deployment anyway, unless the scripts are modified appropriately too).

'devops' means I'm going to hire you ostensibly to develop software, but in reality thats just going to be your '10%' time if there aren't any operational fires burning too brightly.

“Devops” means build engineering to some

Sysadmins who know a bit of python to others

Developers who can configure nginx to others.

Or a culture; to yet more people.

Clearly it doesn’t have meaning if it’s so undefined.

FWIW the progenitor of the word defined it as “agile systems administration”: what you’re talking about is the 10+ deploys a day talk from flickr; which doesn’t mention devops at all (despite the conference existing prior).

If you are hired as DevOps, or if your organisation does DevOps without any dedicated personnel, and all the devs are now DevOps, you are damn right that everyone'd be fighting all the fires first.

The idea being that the people building and running the application are incentivized to prevent fires as much as possible.

As opposed to completely separate teams, where the devs have zero incentive to prevent fires. Everyone judges them on features/sprint, so optimizing for that is perfectly logical.

That's not been my experience at all. Developers absolutely do need to be roped in to help put out fires when the application is misbehaving, and most of us quite assuredly want to keep that to a minimum. And having reliable, smooth and well-regulated DevOps processes is a huge enabler for ensuring robustness and minimizing the chance of bad code getting deployed to production.

What percentage that works up to is totally up to you and that’s the whole point

I think what it is is that nobody wants to force more responsibilities and complexity on Developers so they hire DevOps people. Then the DevOps systems are so complex that it needs a ultra high quality engineer to run them.

The whole seed of the DevOps movement was that developers needed to do more or the company would fail. Over time, management lost their conviction when developers push back and didn't want to risk losing devs so they "outsource" the devops skills.

Not hiring a dedicated DevOps resource and making your developers do it. Guess what? You've just made your developers do operations.

The work doesn't go away just because you've shifted it. I've seen those places too and worked in some, the developers aren't very productive let's just say.

If the dedicated guy helps devs write deployment scripts, write monitoring scripts, set up backup-restore-verify cycles, etc.. then it's devops. if the devs proclaim that the devops guy should do it, then it's just the old siloed workflow again.

note, that the old flow was not a total shitshow with absolute zero productivity ... it worked for quite a while in many places, but it was bad enough in enough places that a whole "movement" grew out of the recommended solution. it's about keeping the communications/coordination/responsibility-tennis overhead down. sometimes that's best done by saying that you deploy what you wrote in any way you see fit but here's the SLA, and so on. sometimes it makes sense to create infrastructure teams and let dev teams use internal tools to deploy, sometimes this require experts at the team level, sometimes not. and ... of course this can be implemented in the most employee hostile possible way and sometimes in better ways too :)

Reply on news.ycombinator.com