Hacker News Reader

Comment by rubiquity | Hacker News Reader

DevOps is dead? Well I guess that means I can finally delete my fat repo of Ansible, Terraform, and Packer configs. Using AWS, GCE and others doesn't get rid of DevOps. You're just managing computers in someone else's data center instead of your own.

View on HN

>DevOps is dead?

Fyi, the title is using the rhetoric template of:

  <SOMETHING> is dead.  Long live <SOMETHING>. [1]

...where <SOMETHING> is deliberately the same word in both sentences. The construct is a shortened version of: <SOMETHING> in its previous and more well-known form has evolved to <SOMETHING> with a new variation. That <SOMETHING> is still there the whole time -- which is sort of meta-joke connecting the 2 sentences.

It's as if someone wrote "Cars are dead. Long live cars." to signal the change from humans manually driving the cars to AI robots doing it. If you read the "cars" literally, it looks nonsensical because there are obviously cars existing both before and after the shift to automation.

[1] examples:

http://www.pcoitmurphy.com/_books_are_dead__long_live_books_...

https://www.tnooz.com/article/hotels-Airbnb-rented/

http://www.wired.com/insights/2014/08/moocs-are-dead-long-li...

I would still disagree with that because DevOps really hasn't changed at all. It's not like AWS popped up yesterday. It's been around for a decade.

To add to this informative post: The original form of the phrase is "the king is dead, long live the king".

It was used when the king died in the sense of "the [old] king is dead, long live the [new] king".

So in that light it means "[the old way of] devops is dead, long live [the new way of] devops".

https://en.m.wikipedia.org/wiki/The_king_is_dead,_long_live_...!

I agree.

I recently created a private cloud infrastructure on Digital Ocean. It would have been easier and faster (and more polished) if I'd gone with AWS, but then I would know how to use AWS, and would not know how to do these things myself.

Why learn about HAProxy, keepalived, tinc, etc when you've got AWS to handle it all for you?

Answer: Because relying on a service to abstract the inner workings for you is a path to the dark side.

Agreed. I wouldn't be on AWS if not for the fact that they're the only large, on-demand instance creation service that will also sign a BAA for our HIPAA needs. Even though we make use of the AWS specific tools like ELBs and VPCs, we still configure all the under-the-hood equivalents such as iptables, nginx, etc. ourselves.

WHY?!?! Just go to beanstalk and have it done for you or use Puppet/chef to manage it for everything.

I'm not sure if you interpreted that as configuring it all ourselves as in manually, but we use Ansible for configuration management.

Not sure why you're being downvoted. Abstraction is for ease of use, not ignorance as to what's occurring under the hood.

No. Abstraction is for higher level thinking. Dev Ops is still young so we are seeing many attempts at providing these abstractions but it is a very large domain to abstract over and thus hard to make a non leaky abstraction.

Given a non leaky abstraction, ignorance of whats under the hood is just fine.

What I was trying to say was more along the lines of a 3rd-party service abstracting the work is not a good idea in the long-term.

It is unthinkable right now, but what will happen with Amazon/AWS goes bankrupt and no longer exists? All these AWS Certified Engineers who are not actually devops will suddenly be lost.

On the flipside, if you learn everything from the ground up and then jump on AWS or whatever service is available, you will be in a much better position when the day comes to move away and to something else.

Abstractions in something like programming languages is fine because the language won't simply disappear one day - you'll evolve as a programmer and will be stronger for it.

> It is unthinkable right now, but what will happen with Amazon/AWS goes bankrupt and no longer exists? All these AWS Certified Engineers who are not actually devops will suddenly be lost.

Or they could just read this document and move to GCP.

https://cloud.google.com/docs/google-cloud-platform-for-aws-...

It's all virtualization/containerization/object|relational storage, no matter how you cut it.

Know the fundamentals, and you'll be fine. Everything else is rhinestone marketing.

I see what you are saying, and I agree with you that betting on any of the current abstractions is probably not a good idea.

But I really believe someone will figure this problem out, and at that point Dev Ops will be done. There is just to much money to be made.

> Given a non leaky abstraction, ignorance of whats under the hood is just fine.

Minus vendor lock-in. Once you have your infrastructure dependent on ELB, S3, Redshift and all the other AWS niceties, you are effectively locked into Amazon. Locked, in the sense that migrating out has a relevant knowledge cost (deprecation of knowledge of AWS, need for gain in alternative knowledge).

It's not a dumb choice, but it's also not clear for either side. Running your own iron is much much cheaper. For my case (email service provider), our bill estimate on AWS would be about 300k€/year, while running our own servers costs 84k€/year (split at 48k€ depreciation and 36k€ operational costs).

> It's not a dumb choice, but it's also not clear for either side. Running your own iron is much much cheaper. For my case (email service provider), our bill estimate on AWS would be about 300k€/year, while running our own servers costs 84k€/year (split at 48k€ depreciation and 36k€ operational costs).

Something to note about US orgs: New depreciation rules allow you to depreciate $500K/year in hardware, in that year. That means that gear is yours, forever, whereas that $500K/year in opex to AWS is recurring forever.

can you link to a detailed analysis of that?

Did you want a link to the new depreciation rules? Or a specific scenario?

Link to the rules, and/or some analysis by a well-regarded accountant. I'd like to route the information internally. :-)

https://www.irs.gov/publications/p946/ch02.html is the relevant IRS publication.

Ignorance of what's under the hood is also called vendor lock-in, which is great for vendors but I can't see it ever being good for clients.

> Abstraction is for ease of use, not ignorance as to what's occurring under the hood.

I'm stealing this. I love it!

I use cloud services because there's only one of me and I would rather be developing features. They also let me think at higher levels of abstraction.

I also like to think of using a cloud service as "hiring" the entire staff working on that product + benefiting from all of the accumulated knowledge that's been baked in. That makes a cloud service a huge bargain to me.

The key is to focus on the services that you can add value to over and above what the AWS managed services offer. There's no reason to futz around with building and maintaining a load-balancer infrastructure if Amazon's ELB does everything you need (unlikely, I will admit). AWS and other cloud services just provide an additional set of tools that may or may not be useful.

It depends where you want to stop the abstraction, you don't write assembly code or your own database/web server, you are just drawing an arbitrary line on products as a service (and I'm not disagreeing with the sentiment).

I'm interested to learn how you setup a vpc on digital ocean considering they don't even offer that.

They do, but it's very limited. Their private IPs only work within the same datacenter, so you lose the benefit of distributing your infrastructure globally.

The solution that worked well for me was using tinc to set up a VPN so all my servers, no matter what datacenter they're sitting in, can communicate with each other using private IP addresses (tinc has added benefit of encrypting, which DO's private network does not).

If you're needing help, I'm currently looking for more contract work :) Check my profile for contact information.

But you are still sitting on the same network as everyone else correct? That doesn't sound like a vpc.

This is a case of the wrong tool/service for the job. You need advanced networking and spent all this time duct taping it together on DO when you could have a real VPC on aws with security groups, subnetting etc. Not to mention locked in to DO's limited instance types.

Don't get me wrong, DO fills a niche for quick one-off type applications but it is absolutely no substitute for a real IAAS like aws, gcp, azure... Investing time in trying to turn it in to one is a waste and way worse than perceived "lock-in".

You also could have just as easily rolled your own setup the same way on AWS, so i'm not sure what argument is other than cost. (btw t2.nano instances are $4.75 a month) http://www.ec2instances.info/?cost=monthly

Plus also: $$$$$$$$$$$$$$$$$$$$

The article is muddled. It's like some came up with a title and then tried to support their conclusion with rambling thoughts without much cohesion.

Deadlines, I guess.

You definitely need to delete your Ansible, Terrafor, and Packer. Typical Linux distro is a lot larger than your project, but it works fine without that, at hundreds of platforms. Did you learn how to achieve such level of flexibility?

I don't understand the correlation you're trying to make. Linux is very sophisticated but it doesn't provision itself.

Linux distro is much more complex than application, but it installation is simple, because distro is split into lot of independent packages, which are wired using package manager. Lot of naїve developers are trying to solve same problem using imperative tools or develop their own package manager instead of learning of existing, well proven solution.

I'm pretty sure lots of developers are also using package managers, and sometimes private servers for those same package managers, but configuration management is still a separate thing that needs to be done and using tools to do it is very much a good idea.

I use same tools (package manager, services, scripts) for about decade. Tried to use Puppet and Ansible, but result is much more complex and it feels like I reinventing the wheel, because I need to implement things which I already have implemented in package manager, like validation, upgrade management, etc.

Difference between Ansible and package manager feels like difference between procedural programming and OOP.

Difference between Puppet and package manager feels like fight between two competing object oriented models.

Far be it from me to tell you where or how something should be implemented. If you're happy using packages then that's great! But the other two tools you mentioned using – "services, scripts" – are exactly what a lot of people find to be harder to manage than a configuration management tool.

Maybe your "services" and "scripts" are really sophisticated. If true, then perhaps you've implemented your own configuration management tool. If so, maybe you could share it and other people would find it useful too.

You are asking wrong questions.

The reason, why I don't use configuration tools is because I do no configuration in production. One of main rules for me is to perform all installation and configuration steps in controlled development environment, then test them in staging and only then deploy verified result to production.

Imagine that you are fully installed and configured everything at your new production host but in your development environment, then it powered off, deployed to production environment and powered on again and it works. It is result I want to achieve: deploy only ready to use software to production, in sealed form, so no installation, no tweaking, no anything. Like smartphone on contract. It can be achieved with docker, but docker is relatively new invention and my solution plays well with docker too.

So, no, my scripts and services are not a configuration management tool.

Oh you mean running your own apt/yum mirror? I've done that in the past. There's still configuration involved. You have to configure the packages that your machines then pull. You also have to configure the mirror itself.

For dependencies, use metapackages. For repository configuration, use "release" package with description of repository. Release package can be installed directly from http.

In my cases, I define few variables, then I will install repository definition and main package:

  export NODE=2
  dnf install -y http://..../staging-release.noarch.rpm
  dnf install -y node-backend

(and few other steps in some cases, e.g. putting of host names into /etc/hosts, or formatting and partioning of hard-drives).

It works fine on bare metal, vagrant, docker, aws, etc.

And how do you tell 100s of hosts to install that `node-backend` package?

It is called provisioning. How you created 100s of hosts in first place? Use same tool.

I think you are mixing the concerns of configuration managers and package managers. They are at different abstraction points. Step one: install software and its dependencies (package manager). Step two: Have all the software services in your cluster talk to each other (configuration manager). Step three is profit.

No, I am not mixing them. I just skipping step two, because it can be done automagically after step one.

It can. Is it efficient? No. This is so obvious that I'll have to attribute your opinion to lack of experience. Try deploying a stack with a dozen different services, designed with fault tolerance and horizontal scalability in mind, where each service is clustered, without a configuration manager. Then, review your opinion.

I have 25 years of experience. 19 years with Linux alone. Yes, I deployed stacks with dozen of different services. Typical distro has hundreds of services.

To anonymous coward, who downvoted me: if your large and "complex" application will be integrated into distro, so you will be able to apt/yum/dnf install it, will you still need your custom installation scripts?

`apt-get install nginx` only installs the package and some default configurations. You still need to manage the actual config files yourself.

What will you use? Bash? Python? Perl? Why not something like Puppet that keeps track of changes it needs to apply and has many useful features on top of it?

What happens if you write your installation scripts to use Ubuntu, but the order comes from high up to move to CentOS/RHEL? Did you write your scripts to be distro-agnostic? Because Puppet's `package` is.

BTW: So nginx developers are using Puppet to install nginx? Because Puppet package is platform agnostic and we have nginx at every platform. (Joke).

I would hope that they have some sort of automation set up to test their code on multiple distros, yes.

You are misunderstands relationships between authors, packagers and users. Authors pays by code, packagers pays by maintenance, users pays by testing and with bug reports and patches. Testing of nginx at all distros and in all usage cases is not responsibility of nginx team. If you disagree, then demand your money back.

Actual configuration can be installed using `apt-get install ngingx-site-config-prod`.

Now, try adapting that approach to do Blue/Green deployments [1]. Imagine how that is done using configuration managers. Can you see the advantage now?

[1] http://martinfowler.com/bliki/BlueGreenDeployment.html

I use it. Blue/Green deployments is one of main reasons to use package managers, because with package manager I can prove that two deployments are identical, including configuration files, which are also deployed using package manager. It was primary reason why we switched from configuration scripts to packages for provisioning and configuration.

Except that you'll need to reconfigure parts of the stack post-installation. You consider that an upgrade of the configuration "package"? You're reinventing the wheel. Doable, not efficient.

Why I need to reconfigure parts of the stack in first place? I am devops, so these things are automated.

In general, plan your changes ahead of time, test and automate them at staging servers, prove correctness of changes, and then deploy to production. Never do anything by hands (or by configuration tools) at production servers.

Even if you write RPM/DEB's to install all of your config files, you'll need something to trigger installation after you've updated your packages. How are you going to do that? SSH to every server? That doesn't scale very well. This is where other tools come into play.

Reading this thread makes me feel like I'm looking at a conversation between sysadmins from now and one from the past. I'm not saying this to be dismissive though. It's possible there may be merit to what they're describing, maybe the emperor really isn't wearing any clothes. It just takes me back and feels weird to read.

If you are old enough, you should remember times when we need to enter network configuration manually, using network configuration TOOL: netmask, IP address, router IP, DNS servers, domain. Then DHCP SERVICE is invented: DHCP client asks DHCP server for that information and applies it automagically: it just works, for any device, now and in future, without human.

Ops tells me that they are using configuration TOOLs. I tells them that I am DevOps, so I use SERVICEs and SCRIPTs (clients) instead. They work automagically, without need for me to reconfigure servers manually.

Am I from the past?

Would you be opposed to publishing your architecture on github with minimal examples? The examples would need to be enough for someone to test out and verify for themselves. I think that would help people understand and contrast this against other solutions.

One-time service and reboot of host or start service manually from kick-start script.

Reply on news.ycombinator.com