Hacker News Reader

Comment by lykr0n | Hacker News Reader

This is true to some extent. I'm building out the infrastructure for my company, and there is a lot of effort on my part required to automate stuff.

Something like kafka requires figuring out the configuration you want, putting that into Salt, adding the correct configuration options, deploying it and making sure zookeeper works, and then generating certificates and what not. It's not a simple process.

Setting up monitoring, and other things like floating IPs is a pain. Custom wrappers for Terraform scripts and other components required to deploy the systems you need to run an app. It's a lot

View on HN

Ugh this comment triggers my hate for the current state of infrastructure. It's just configuration hell at this point. We thought all of these wrappers and services would make infrastructure easy, and they did to a certain degree, but it also created brand new problems that are arguably just as annoying as the old ones.

Edit: As a developer, I would rather sketch out the infrastructure that I need, then hand it off to someone whose entire job is to set up infrastructure. I work at a small startup where the devs are still doing a lot of infrastructure, and all it does is create technical debt. Without a solid team that owns infrastructural concerns, developers just end up digging themselves a grave.

I think the problems are mostly around configuration that is so complex and redundant that we need programmatic abstractions to deal with it—we need something like a general purpose programming language to DRY up and simplify that configuration, but the powers that be are clinging desperately to YAML (despite encoding a shitty, half-baked AST on top of it a la CloudFormation or “generating” YAML with text templates a la Helm) as the human/configuration interface presumably because the “it’s as easy as YAML!” is such a good marketing schtick.

A certain amount of configuration complexity will always be there, but there’s still a lot of incidental complexity that could be cut away if we just generated these config a with a general purpose language.

HCL2 feels like the farthest ahead right now, if you don't want to deal with straight up python or ruby.

In case you haven't seen AWS' answer: https://github.com/aws/aws-cdk

There are worse things than YAML. Dealing with hundreds of snow flake configuration formats that do not follow common sense rules cost me more time than working around mistakes in YAML. There are obviously some really, really stupid ideas like generating YAML with text templates but the problem isn't with YAML. It's the tool that generated the YAML file.

It doesn't feel like we've come very far at all from the first time I realized I could use nginx as a forward proxy and have the code just make all service calls to localhost. That seriously reduced the complexity for the development team.

Some days I just want a load balancer that can route traffic to whatever servers are up now and then I'd just have Docker, a service registry, a load balancer and I'd be done for a while. The easy bits should be easy, and they can't even manage that.

That's pretty much the experience I had with Consul and Traefik (containers orchestrated by Nomad).

Isn't that the point of the article? That infrastructure actually isn't easy, and you're just hating the fact that it isn't?

Isn't that sort of like someone in Sales complaining about the "state of development" and asking why they can't just press a button and have a custom app to sell, like, tomorrow?

Welcome to dev vs infrastructure vs ops vs devops.

You know that your job has abstracted into insanity when use the names of products but the sentence doesn't read like it.

Kafka requirements adding salt to keep your zookeeper happy.

JavaScript went into this stage not too many years back. "I test my react code with mocha chai cucumber in gulp with puppeteer."

In this case, I'm not sure if it's abstraction-insanity as much as having common solutions to common problems. You know what would be an insanity of abstraction? Rolling all that yourself.

Given that there are better and worse abstractions, most are not suitable for most use-cases, a lot of times simpler and more centralized is actually more stable. But when you start needing high-availability-coordination you're going to be damn happy that there are things such as Consul, etcd and Zookeeper.

As for Salt; by all means, start off by having bash scripts in source control, which at some points becomes a pain-point and you'll realize you need some kind of configuration management to not go insane.

Several of these tools are just putting a framework and common language for tasks we have to do either way.

Isn’t this stuff supposed to be solved by products like DC/OS?

DC/OS is dead, Jim. Well, kinda. Mesosphere changed their name to D2IQ and is now supporting kubernetes.

You can, however install the banzaicloud kafka operator with only a few commands and very little config and yield a production-grade kafka cluster pretty quickly.

https://github.com/banzaicloud/kafka-operator

Reply on news.ycombinator.com