Hacker News Reader

I think this is a common fallacy.

To randomly pick an example - sure, automatic filesystem snapshots are a cakewalk these days, and a decade ago they were rather expensive. It seems logical to assume that things that we needed admins to do and rig up somewhat delicate systems for a decade ago are so easy now, we don't need people to focus on that...

This overlooks the fact that the baseline has just moved ahead. Sure you don't need dedicated people for stuff you used to.. but there is new stuff out there that your competitors are hiring dedicated people to work on and push the envelope. if you're okay with doing what you could have done 10 years ago, just using reduced staff, that's great - but it's not going to win you much.

Is operations really going to be a competitive advantage for you? If my startup lets you hail cabs from your smartphone, is an extra 9 of uptime really going to help me beat Uber?

My company's space is pretty crowded with competitors, but we're beating them because our service returns results in 78ms, while our competitors doing devops and running in Azure, etc. return in 2500ms.

That's because we have a dedicated ops team who concentrate on performance and scaling, and leave the devs to write code instead of managing servers.

Surely the devs focus on performance and scaling as well, if not at the level of managing servers? Having a dedicated ops team is a necessity, but it's important to keep developers in the loop on operations as well.

As the architect of the systems, I focus the designs on scalability and performance and the devs implement it. So, whether it's a dev or its an architect, someone has to be thinking about it. But having an ops background helps with knowing how to scale.

  "I'm a sysadmin by profession but when I write code I don't
   want to have to worry about backups, scaling, etc. Those
   things get in the way of creating the product."

This is how you get yourself into the type of problems OP is talking about. You can't rely on infrastructure to intelligently save you. You can't rely on SAAS options to solve your design issues. If you aren't asking questions from the beginning like:

"How will I scale my product?"

"How will it handle failures?"

"How is it going to work at scale?"

Then you're going to be dealing with complex troubleshooting issues in a fragile infrastructure.

Don't try to replace the engine mid-flight.

Just because I don't want to worry about it doesn't mean that I don't. In fact these are the things that keep me employed. The point I'm making is, the level of abstraction is increasing and putting us in a place where someday we wont have to worry about them. It's the same way as people can now create websites without needing to hire someone to build it.

Actually it gives you more to worry about. When something blows below the layer upon layers of abstraction you will be helpless. Ask anyone that tried to find workaround about a bug in a core closed sourced framework. You are suddenly playing code permutations.

I have seen the same with some of my admin friends when they throw config after config in a case where something is not working while the documentation says it works and reality says it don't.

So it will hit the fan less often, but when it hits - there will be more than enough for everybody.

I'm a programmer by profession, with some admin experience, and I couldn't disagree more.

It's entirely possible to write code in ways that make it really difficult to backup or scale and it's your job to avoid this and any other gotchas.

An obvious example, would be taking a binary copy of a database as a backup means. I've seen this a ton of times, though most good admins know there are better ways to back them up. How do they know this? I'd hazard a guess because the programmers were aware of the problem, documented it, and wrote the tools to get around the issue.

I imagine you really meant to say "I don't want to manage...", which is fair enough as long as someone else is doing it. :)