From my perspective in doing infra / devops stuff, they exist to make developers lives easier and faster (CI testing and continuous deployment), or in the case of monitoring / platform engineering, looking to "enhance" their capabilities. Meanwhile the people who do both feature development and the bugfixes are product developers ... So in my eyes it seems like the "auxiliary pieces" like devops people wouldn't be seen as core or necessary to the business in the eyes of managers looking to lay people off
Older companies with poor engineering practices don't have devops people at all, often times they don't even unit test actually, maybe it would be a riskier position to be in charge of those pipelines
Perhaps the data point of Meta still hiring SREs is just one data point as well (they could just be in more need of SRE people), although it is so hard to find more market wide data
In 07 I was very young and worried about the economy; when expressing my concerns to an older coworker said "dont ever worry, we've always been and always will be in demand". He was right. I jumped ship 3 times during the last recession, all with big bumps in pay. I've never worried about finding work, so much so that if pressed I know I can secure 2-3 offers after a week of looking
But to answer your direct question I think its three fold:
* No one wants a junior in production so unless you're well established and can burden the cost of one good luck getting in
* The educational path is very informal and unless you're going for this type of work specifically you're not learning what you need to be
* Most SWE's dont make good devops (sorry thats a fact no offense meant to anyone)
New abstractions come along (eg. AWS), bringing additional complexity, then better tooling gets developed to deal with it, and this prepares the way for new abstractions on top (eg. Kubernetes).
At the same time, legacy systems remain around a long time and need experienced people to maintain those.
While operators are glad they don't have to deal with bare metal any more, there's a lot of additional complexity you need to understand to manage AWS effectively, that didn't exist for most companies back in bare metal hosting days.
It’s no longer acceptable for anyone to turn their service off for maintenance for example. They all have to roll updates out frequently and without downtime. They all have to scale to huge numbers of users and data. They all have to have almost perfect uptimes.
and they turn off their game everyday for 10mins :P
- DevOps only, as in Terraform or CloudFormation or CDK but not much else
- DevOps/SRE/DX, where you have engineers who handle DevOps as part of a mix of Platform Engineering, where your Platform team is able to guide and assist Product /Experience and Data teams on every part of the Developer Experience, from infra to on-call
I think it's the Platform Engineering model that has the most value; I don't know that DevOps without the rest of the package is as valuable
Fast forward to today where everything is running on servers and you're now responsible for installing, updating, monitoring, and scaling yourself, and probably also want some level of automation in the test and release processes. The other half of the software lifecycle is now core to your own business. The engineering challenges posed and the skillsets to handle them overlap with product development, but are not identical. So specialized roles have evolved to fill the gap.
As for the apparent demand, this is the part of the software lifecycle that CS and SE degree programs largely gloss over or don't teach at all, and there are no bootcamps for it. It's also very difficult to self-teach for the same reason there aren't a whole lot of self-taught high-energy physicists. It's easier to build out a homelab than a particle collider, but largely you need equipment beyond a personal laptop. This means most of the people doing this kind of work learned on the job at some point, and that means they're more senior and rarer.
--Dinosaur Unix SA