Think boiler room vs drafting room.
Think boiler room vs drafting room.
Provisioning servers, databases, load balancers, etc and other resources use to involve someone driving down to the colo and installing hardware. Often now, it's a yaml file you run through Cloud Formation.
Devops is just as much development and software engineering when you are working in a cloud environment -- it's an API call.
Certainly, in other engineering fields, visits to the installation sites by (designing) engineers, or even running pilot programs to debug the system as designed and implemented is a norm. However, the notion that 'an engineers job is to be responsible for x-widget from design to operation' is entirely foreign to any notion of "Engineering" that I am familiar with.
How many hardware based Kickstarter projects have failed because they didn’t understand the complexity of actually manufacturing there product at scale.
It’s analogous to software engineers who don’t appreciate ops. Writing software for a company is useless if you can’t ship it or deploy it. You can’t imagine the number of bright eyed bushy tailed “software engineers” I’ve dealt with that thought they were smart because they knew leetCode and learned algorithms who couldn’t troubleshoot performance issues or understand why they couldn’t get shipping products because they didn’t understand the ops side of things.
Of course I am not diminishing the role of software engineering, I’ve been developing software actively professionally or as a hobby for over 30 years. But, I know enough in theory to talk to the ops guys for on prem/colo/hybrid architectures and I could easily be competitive on the ops side for jobs dealing with fully “cloud native” solutions for a small to medium size implementation.
I think you are missing the point.
This attitude is the ultimate reason for dysfunctional development teams and the source of the barrier between operations and development.
It’s also toxic.
Tell me that is not software engineering, the only difference is that my ability to test before deployment is much more costly and limited, I have to get it right the first time around..
And if that’s not “real programming” sometimes you actually do end up writing custom resources in a “real programming language” that get called from CF to create resources.
I’m very much up and down the stack any given week going from the front end, middleware, databases and infrastructure not to mention building out CI/CD pipelines.
Does anyone spend any time thinking, researching, planning, designing or communicating?
Creating these strict barriers between teams, especially with the condescension you elicit is exactly how problems arise.
Suppose software engineers write an O(N^2) algorithm because they didn't think the data would grow large. It does. DevOps then launches many more dozens of computers to handle the algorithm. If no one speaks to eachother, everyone thinks things are going smoothly when in reality it's a complete disaster. The DevOps team has much more visibility into the usage, performance, and cost of apps than many developers. Talk to your breathen wisely.
Probably most importantly, my experience on DevOps has been a bit like hurried product management and software engineering work--I have to identify problems and opportunities to improve software development efficiency, uptime, etc and find/build technologies that solve those problems, and all of these technology choices are really complicated because they are interdependent. How you structure your CI/CD pipeline depends a lot on your orchestration platform (and vice versa). It also depends on whether you use micro services or a monolith, and whether your platform is mono/bi-lingual or fully multilingual. And unlike product management or software engineering, you have to deliver value in a much tighter timeframe (you don't get to spend a year building out a solution with budget and resources for user testing, QA, etc). Further, if you think the frontend web stack is changing fast, the DevOps world will make your head spin!
Regarding YAML, it's just the (currently popular) interface; it doesn't say anything about the amount of complexity. It was originally YAML/JSON because everyone thought it would be super simple configuration, but virtually every infrastructure-as-code (IaC) tool is moving away from YAML to something more powerful.
AWS has CloudFormation which began as JSON/YAML with features hacked on to support things like referencing other nodes in the document, defining and calling functions and macros, evaluating conditionals, looking up values in maps, etc. Now they're building the Cloud Development Kit (CDK) to let developers write IaC in languages like Python and TypeScript, because even the extensions to CloudFormation are insufficient (no ability to pass around parameters, pass structured data, perform computations, etc).
I don't use Terraform, but I understand HCL started out a lot like YAML but is also becoming increasingly powerful with each release.
Helm also started with YAML but is adding in Lua.
The argument that DevOps is easier than SE because it's just using tools created by SEs is silly on its face. First of all, those tools were _implemented_ by SEs but created by much larger teams, including product managers. Secondly, "SEs" are also just using tools created by SEs--compilers, interpreters, VMs, build tools, etc. Not to mention all of the tools and processes built and/or assembled by DevOps engineers.
I use to think that all “true” developers needed to know the ins and outs of assembly because every time I went to a new platform that’s the first thing that I did - learn assembly language (65C02, 68K, x86), but then I grew up.....
CloudFormation takes a text file and generates API calls much like any code compiled down to assembly.
I believe you would benefit from a different perspective because this vision is partial at best. The distinction between who builds system and tools and who uses them is subtle and often these roles are interchangeable. Both have dignity and purpose and contribute to the goal, regardless of the collar color, which is a remnant of the last century.
Galileo built his own telescopes.
People who are really serious about software should make their own hardware.
That is no different from people who call themselves “software architects”. You can’t build great software architectures if you don’t know how to build and optimize the underlying architecture it runs on and how to build the pipelines that put the code on those systems.
i deploy all our cloud instances using Ansible (yml files) and I wouldn't have a hope of doing that if I didn't understand the OS I'm configuring with it (Linux).
I might see your point if you're talking about the yml that is used to configure 'value added' products from cloud vendors, but I avoid those like the expensive handcuffs that they are.
This leaves me in complete control and keeps prices low because I only ever use vanilla instances.
I'm not on Amazon right now, but when I was I used to use ansible to configure networking on it and security groups etc, but I would always try to keep the vendor specific stuff to a minimum.
But I never said I was using Amazon.
I'm using an internal cloud service at my company. It does not offer any managed services.
Also when I do use AWS for clients, I _sometimes_ use managed services¹, but most of those lock you in to Amazon. Some clients don't care, some do. They are paying, so I don't care either way.
In either case, I spend very little time babysitting infrastructure, just FYI, even when I use bare instances.
And in my opinion² if people aren't careful about managed services, then they won't have any OS knowledge anymore, and will just be left with product knowledge.
Sure, managed services make things easier, but they also make your sysadmin muscles weak.
1: Aurora DB for example. That one I like because there's no lock in. I can always move my tables to a standalone MySQL.
2: As someone who is 100% comfortable using bare Linux without having to use containers, managed services, control panels or even google searches to keep my stuff running, and have been doing so since the early 90s.
I want my sysadmin muscles to be weak. I’m not saying that sysadmin isn’t important, but why spend time managing servers and doing the “undifferentiated heavy lifting” instead of adding value and dealing more with strategy?