It Was Never About Ops
biven.org
biven.org
Is this why Silicon Valley is allergic to the phrase 'systems administrator'? Ops, Devops, SRE... so many phrases to describe various jobs which all, when executed successfully, converge upon the same set of tasks, and the same approaches to them, that has been labeled 'systems administration' for thirty years or more.
What is the motivation behind this trend?
As I have more control over hiring decisions on my team, an inability to use any programming language effectively (bash, python, perl, /anything/) would disqualify someone from a job in sysadmin or information security.
The focus on automation, and it being called DevOps, is simply an evolution of the work of sysadmins. The same way devs evolved toward TDD and other modern programming techniques.
I believe you have this exactly backwards.
The problem is that we keep silo'ing everything and inventing new buzzwords and half-assed tech. We can split things into smaller pieces quicker than we could ever staff up.
Nothing has changed since the 80s. The only thing that's different now is that instead of programming in one little pond of a language, now everything you do is programmable. That doesn't mean you need 47 specialists. It means you need to get really good at managing and limiting complexity.
We don't want to split off and evolve roles into newer and even more specialized roles. This is the thing that made deployment and updating so whacked to begin with.
</rant>
Ops team may create an automatic solution, e.g. script to call with parameters and a runbook. DevOps will create fully automated solution, without need for a runbook.
By definition, DevOps is developer, (an one who uses Agile techniques, such as automated test cases, CI, source control systems, ticket management systems, etc.), who is able to do Ops tasks with code.
> DevOps is a set of practices that emphasizes the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes.
DevOps certainly does not mean that Dev has to do Ops now, too, although it's frequently misunderstood as meaning that, and IMO that usually leads to a lot of tears in the long run.
Wikipedia quotes description of tne conference, not a description of DevOps role. However, if you read carefully original article, you will see that main idea was to close gap between ops and dev team by bringing dev techniques, such as Agile, to ops team. So, DevOps IS developer with Ops knowledge.
This condition has held true as long as there have been systems to administer. Only the names have changed; that's what I'm curious about.
There is no need for 'systems administrator' in the Silicon Valley.
The systems administrators are the guys who buy the desktop computers, setup windows, and kinda manage the active directory.
DevOps/SRE names exist because the tech scene had to stop the confusion between the hardcore linux admins who can code and the dude who can setup your desktop computer. The first is needed in dev companies mostly located in the tech hubs, the second is needed in most companies all around the world. The two have very little in common.
If that confusion existed in any otherwise-competent organization, it was only in the valley. I was a systems administrator before any of "SRE," "DevOps," or "linux" came to be, and I don't think anyone has ever confused me for tech support.
Several years on, I still don't know what to think about Powershell, it's basically .NET + pipelines + some discoverability aids, not terrible in concept but I will probably go on ignoring it for the rest of its life.
The places where I have seen DevOps become popular are those that completely separated programming keeping servers running, and where the people keeping said servers running were incapable of doing even mild automation. Places where rebooting a non-db server requires making a request 3 weeks in advance, and being OK with 4 hours of downtime.
In those environments, those people that are called system administrators are very low productivity, can't help you with a performance problem if their life depends on them, and are easily replaceable with shell scripts. It's in those environments where DevOps and enterprise cloud migrations are popular: The developers might not be experts in scalability or in getting great uptime, but at least they can get something done. Ultimately this gives management an excuse to lay off all the sysadmins that don't know what /proc is, and make technical decisions based on Gartner reports.
Again, that doesn't mean that everyone that has the title of sysadmin in the world can't code or is unproductive: This is not a judgement on the value of a good SA. I am just trying to open a window to a world that many developers get to see, but few good SAs notice, because there's no way in hell a good SA would work in said companies for long.
There are places (places with a lot of money on the line) that do not fall into these categories, and yet the solutions are not so different. I'm just intrigued by the level of effort Silicon Valley is willing to exert to avoid recognizing this -- or even admitting that it's possible.
For example, in places I've worked there was a "definition of done" where in order to have a new service be taken on by ops, it had to meet certain criteria so that the people running all these services had some common basis for diagnosing and running them. You don't want 100 different ways of calling a healthcheck on a service. You want service nodes to be able to handle a hard shutdown without data loss, maybe. Stuff like that
It's actually pretty fun to have an ear in a wide variety of businesses, you wind up learning a lot outside of the technical world.
Is the extent of your agreement service within a day or something shorter/longer?
Even with a Gold support plan with 2 hour guaranteed on-site response, it took nearly 2 days to get our servers replaced and running, but the bigger guys were back up within few hours of the outage. Sun simply didn't have the staff or immediate spares available to move any faster -- their response was still impressive - they brought in an 18 wheeler full of spares the next day.
They ended up giving us some months of free support or something like that to make up for the delayed response.
I do have a couple of clients that pay quite a bit more for a call-and-fix any time guarantee, but I only offer that level of service to businesses where 1: there is at least one person on their regular staff who is technically competent enough I can walk through more advanced issues and they will understand how to follow instructions, and 2: I'm the one that built out their network, so I know the gritty details.
For the times when the shit hits the fan in two places at once, I have friends who do similar work, and there are a couple of guys I can call to cover if needed. Obviously they get most of the money from these instances, but the customer stays happy, and I get larger monthly residuals than I could otherwise take care of, so everybody wins.
One favorite situation I ran into: a database server with a RAID card set to do write caching without NVRAM on a server with a single power supply (so two bad things in one). An overloaded power strip blew a circuit breaker and when the server went down, the filesystem their database was running on was unrecoverable. Oh, and of course they weren't doing backups because they had RAID!
"Thankfully", problems like CryptoLocker have, paradoxically, made my job as a sysadmin easier, as now most companies I work for are familiar with the horror stories of how someone lost years of work/all their bay pictures/etc and did not have a reliable, cold-storage backup to deal to recover from such attacks.
Titles are a good way for management to increase responsibilities for an employee, without having to pay them more.
DevOps is a developer, who writes code to do tasks which are often done manually by Ops team. E.g. installation, configuration, assigning, recovery from failure, upgrade, migration to new version of database, etc.
'系统管理员'
('xìtǒng guǎnlǐ yuán')
I'm thinking of doing the traditional anglo thing and getting a tattoo of chinese characters I'm not really familiar with. I'll tell people that it means "fiery heart of the tiger" or "blessed heaven spirit" or somesuch... :)
> 1. Expect services to grow at a non-linear rate.
Why would you expect this?
This rings true.