Hacker News Reader

Comment by trey-jones | Hacker News Reader

I might need some perspective. Every time I read one of these I'm just shaking my head, and yes the arguments are Less Power, Less Control, but also SysAdmin is me. That is my identity, and the part of the job that I like the most. I solve problems with configuration before code. So I hate to think that I'm becoming obsolete, of course. I also don't think that I am, because I do think that my way has some clear and definite advantages over eg. Netlify. I also think that "my site is free as in beer and also lightning fast" generally transposes to "my site doesn't have any traffic". Am I deluded? I recognize that I have a clear bias, but even stepping back, I do think that even if some don't need a VPS (or server) anymore, that there is still room for my ilk in the tech world.

View on HN

I would note that every single company I've had in the last 8 years has essentially been replacing sysadmins with infra-knowledgable programmers working with AWS services (who might have been sysadmins in a past life). No one in the engineering org was hired who couldn't program. IT's "System Engineers" were aimed towards "you need to write code to do your job with LDAP/AD/Okta/etc". If someone identifies as a sysadmin and can't write Python/Perl/Go/Ruby, I would... strongly suggest changing that, because I think _that_ has become mandatory for forward looking shops, if not normal shops.

Also. With respect to cost.

AWS costs me something under $1/mo to host my static page on S3 and have my domain point to it. I don't have much traffic. If I did, I could sort that out with relevant tooling; it's still wildly cheap vs compute. The FaaS model (although I personally loathe it) is also reported to be very cost-effective for certain classes of problems.

> [..] If someone identifies as a sysadmin and can't write Python/Perl/Go/Ruby [..]

this is the difference between a systems admin and a systems engineer. some folk can't make the jump from sysadmin, despite best intentions and training.

It is a common (new) myth that sysadmins didn’t code.

There were some IT technicians who couldn’t code and some operations staff who couldn’t code (and would proudly claim this, for some reason). But tools like ansible, salt, chef and terraform were written by sysadmins.

In fact a lot of non-feature code in my companies such as database fencing programs and automated blue/green rollouts has been written by sysadmins.

We’ve just been starting to call them something else since 2013 now.

Some sysadmins could code. Not all sysadmins can code.

Sysadmin long enough and you'll find yourself inadvertently scripting in Bash. A few languages and API later and you'll realize that you may be coding. That is how it went for me...

"DevOps" == *nix admins before IaC

I cut my teeth on ancient VAX, AS/400, Solaris, HP-UX and Irix connected via NFS and Fibre SANs. Runbooks became shell scripts, then a DB backend, etc. Chef/Ansible/Puppet/et al all evolved from this practice.

At the time I moved on to greener pastures, I left behind a global Enterprise Storage environment that was semi-automated by workflow. Directly descended from ksh/bash and then moved to REST once that started to be available. Vendor tooling cost $ and sucked.

"Oh, you need to report on all your disparate proprietary backup environments, success/failure, generate reports and ServiceNow incidents? Cool, we'll just bang that out in python and store in Mongo."

Today, that would be a startup SaaS play (adjusted for technology of course).

It is amazing the stuff you learn and create to prevent burnout/churn while maintaining critical infrastructure as a cost-center org.

This might be pedantic, but DevOps originated in the idea that ops and devs needed to work very closely together to deliver capabilities. It _then_ turned into a role "cloud sysadmin" and then lately seems to be "build engineer".

Language shifts, man. Weird stuff.

Somewhere along the switch from "System Programmer" to "Sysadmin", probably around turn of the century Windows Server timeline, "sysadmin" somehow started to imply there's no programming.

Despite ridiculous amounts of Perl written by system admins, among other things.

What's really interesting is that I've run into a lot of sysadmins who can't/don't code; its a continual surprise to me. Writing Perl to manage a system was just part of the deal back when (dating myself now. :) ).

I think that given people don’t believe now that sysadmins used to write a fair chunk of code; what has happened is people hiring “sysadmins” these days don’t make coding a part of the job.

Thus, sysadmins today are doing what helpdesk and system operators used to do 15 years ago. Which means it is now true that sysadmins don’t code, due to a weirdly shifted perception over time and that perception becoming self-fulfilling.

This is something that annoys me. I’m most comfortable identifying as a sysadmin for a few reasons. I spend a lot of my time on problems that don’t make sense to automate. I spend my time at the code level building automation tooling instead of creating or debugging application code. Historically I have often spent my time with developers guiding them on how what they’re doing fits into a broader ecosystem, teaching them something both critical and esoteric about the application so that they can do their job better, and often guiding them on specs.

It’s not everyone’s experience, but my experience has been that a lot of developers who don’t have a sysadmin background aren’t as skilled at those things. The business analysis, cost control, or the application meets infrastructure parts. Case in point, one of my best friends and frequent professional collaborators is a pure-programming guy who works in AWS. He knows everything there is to know about provisioning the most common AWS resources via CloudFormation and Terraform, but I have never seen him demonstrate a realistic understanding of costs or _why_ certain resources are used in lieu of others, or a strong aptitude in optimizing for specific workloads and business cases. If he hits on an area that is “traditionally” sysadmin work, it’s deer in the headlights.

I was a junior system administrator... And became an early cloud adopter...

I think your viewing is valid, but yes, biased by your knowledge.

Since you already know system administration, it's not hard for you to manage another system... That's your day job, what's a personal one?

But for people who prefer code, handing that off is a blessing.

I runs small light sail vps.. it took me only a few hours to write up scripts to manage containers.. and a few more hours to automate backups to a secure location in another AWS account... But for today years, what I just described is impossible, or a 1-2 week task.. not an afternoon...

And this is our bias... For us, it's easier than learning lambda, or api gateway... But for them, those are easier than learning system administration.

I also disagree with the idea we are becoming obsolete... I work in a serverless DevOps environment for 4+ years and still explain basic networking concepts regularly... Stuff that don't teach much of in computer science... Like failover DNS ... While managing DNS was moved to the cloud, many still don't understand the technical reasonings of things.

You're not obsolete. It's just that some people just want to drive their car and get it serviced somewhere every once in a while rather than doing it themselves. Same for servers.

To be sure, it's not getting it serviced "somewhere," it's getting it serviced at the dealership.

No, some people want to rent from a fleet of cars, so that even if it needs servicing it can be replaced in less than a second.

Or they want a taxi.

> I also think that "my site is free as in beer and also lightning fast" generally transposes to "my site doesn't have any traffic"

its more about being able to handle spurts of traffic and sustained traffic. these services are good enough to power whole communities and mobile apps. and that means it covers most use cases, as well as the placeholder websites that may launch a product once.

and by most use cases, we're talking about just the 99th percentile of high load intertwined ratsnest products not being able to handle it, but even then that's only because of an insanely high traffic load that is loading unique queries that are impossible to cache and this breaks view rendering - aka the bottleneck is still elsewhere.

with things that render quickly and have that view cached on the nearest CDNs, and that have giant memcaches of the queries that will be used, you really aren't needed!

fortunately there are many web services that are not designed that way, cannot be designed that way, but more so can't/won't be redesigned that way.

> I solve problems with configuration before code

I sometimes write code, I sometimes configure things, I sometimes write code to write configuration, sometimes I rearchitect the damn thing. I do whatever it takes to reach the business goal.

If all you know is how to configure something, it's like only having a hammer in your toolbox

If all you know is how to code something, it's like only having a hammer in your toolbox

If all you know is how to run a network, it's like only having a hammer in your toolbox

Not many companies want someone that will move from network problems to code problems to configuration problems to physical problems to supplier problems to security problems over the course of an afternoon, but if you do get a job like that it's nice and varied.

Well I appreciate the feedback, but I didn't mean that I only configure. But over a decade in the business, mostly as a dev first (an infra-knowledgeable dev seems to be the term being bandied about here), I discovered that infra is my first love. So I don't wish to outsource it.

You're not wrong, and I think the right way to think about it is in the context of a company that wants to go "multicloud" -- suddenly they need to abstract a suite of infra-as-code configuration components up out the individual services and APIs offered by various cloud/VPS providers and the reductio ad absurdum of this process is systems administration.

Essentially, a system that requires administration is the opposite (on the far end of spectrum) of vendor lock-in (whether that's AWS, or technologies like k8s, et cetera). Make of that what you will, in terms of the need for your ilk (and mine) in the tech world.

> there is still room for my ilk in the tech world.

Yes, there is. AWS/GCP/Azure are incredibly complex beasts, and people who know how to properly set them up (anything from proper user management and rights to correct ingress points and resource allocation) are as rare as 100-carat diamonds. It's a different kind of SysAdmin job, but it is that.

Don't believe people who are saying "you can just hire junior admins or infra-knowledgable programmers" are talking out of their asses (or until a kubernetes cluster upgrade or a lapsed IAM role brings the company down for 24 hours).

When the winds of change blow, some people build walls and some people build windmills.

You strike me as the type of person who wants to think about themselves as building windmills, but your resistance to embracing technology is building walls.

You’re being a Luddite, afraid of the textile mill because of the time you spent learning to weave. That’s not a good thing for someone who likes technology to do.

Go and learn go, and become an SRE.

That’s all the awesome parts of sysadmin, but modern, automated and without the massive downsides.

There will always be sysadmins in the same way there will always be Fortran programmers. There are benefits to the old way of doing things, but those benefits are of shrinking relevance.

Who do you think runs the cloud you run on? (Hint: sysadmins)

Yep. Just like Fortran runs my bank. Both critical services, and both rumps of employment for a once-widespread speciality.

Don't you mean Cobol? Atleast here in Aus it was cobol on mainframes... My understanding of Fortran was that it found its niche in high performance scientific computing.

Sure, COBOL works for this example too. I was originally going to be a little spicy and use C++, which was once also the "everything" language but now exists only in a few (shrinking) areas.

Yes. As Sartre said: The cloud is just other people's hell.

We call that "abstraction" and "layers". And yes, layers leak, but we still use layers.

Those sysadmins are by and large someone else's problem now :-p

> there is still room for my ilk in the tech world.

There is. I'm trying to hire more of it as we speak.

Reply on news.ycombinator.com