Hacker News Reader

Comment by noisy_boy | Hacker News Reader

Not directly related to copilot but in context of ever increasing yml/json configurations for cloud apps, am I the only one who feels overwhelmed with the configuration options? I see tons of articles with just drop this json here and that yml there and run this cli command - where is the reference for these json/ymls? Does everyone fully know/understands the structure/options? How do people keep track of the changes to that between upgrades? I feel increasingly dumb the more I see these configs.

View on HN

Oof yes. The times I read stuff like "to make sure your WidgyPidy will automatically balance your BoolyPooly cluster, just set the autoEnableBalanceBoolyClusterGenerated in your WiPoBaClConfig.yml file to the desired floating point and drop a reference in the ServerClusteringPoolDirective.json in your configured ShimpyDimpy config folder" like it's obvious and simple...

Finally, DevOps has caught up to the rate of progress in js framework land. Rejoice!

This starts to sound more and more like how they manufacture plumbuses

I always wondered how plumbuses got made.

Me too! Now I know: https://www.youtube.com/watch?v=eMJk4y9NGvE

High IQ joke

This is just.. accurate.

hahaha.. lol.

I feel like trying to use a "dumb" declarative config language for everything is part of the problem.

The domain is not simple enough for that, so you end up with all kinds of weird scripts around it all, or other templating/codegen solutions, plus all kinds of ad-hoc conventions that are extremely easy to break and very annoying to navigate.

Having a full blown statically typed language like TypeScript - where things can reference each other strongly (not just by names and other strings), naming conventions can be encoded in functions (or even enforced by types), and structures on every level (even your project-specific ones) have a well-defined and checked schema - would help a lot in my opinion.

Maybe deno could find a use in this niche.

You're (accidentally, I think?) describing Pulumi in striking detail.

https://www.pulumi.com/

Kind of. From a brief look, it looks very framework-y and all-or-nothing (though I could be wrong, they do have a page about gradual adoption at least https://www.pulumi.com/docs/guides/adopting/).

I was picturing something more low level and less opinionated, though I can only guess how such a thing would perform in practice.

Not even necessarily a specific product, just the adoption of workflows that use already existing tools (e.g. the various API client libraries - for AWS, Kubernetes, git, docker registry, etc.) to achieve this. I realize that that could lead to reinventing the wheel, and eventually an in-house framework anyway, but it feels a lot more comfortable to start there, as you can design things closer to your exact needs.

Pulumi definitely looks interesting though.

It sounds like you're describing HashiCorp's HCL2 and Terraform Configs which are at least a graph.

I find using AWS without Terraform to be nearly impossible.

Terraform, or Troposphere, or cfn-modules/Widdix, or CDK.

Cloudformation as a JSON/YAML vanilla config is just awful, and really exposes the inconsistancies between the different services.

...like CDK?

https://docs.aws.amazon.com/cdk/latest/guide/home.html

Scripts which generate the config also have their own problems. Check out Dhall, it attempts to solve this.

Reminds me a bit of jsonnet (https://jsonnet.org/) which didn't catch on much as far as I can tell.

I see very little practical value in a DSL like this. Using an existing language with an extensive ecosystem and great tooling was half the point of wanting TypeScript in the first place. Most of the "risks" can be handled with just a few eslint rules, tests (e.g. all outputs must satisfy a set of constraints), and code review.

Especially when the moment you need to do more than just generate config files (e.g. fetch some information from an external service, or even write an imperative script for the occasional task), you'd have to switch to a different langauge and all its tooling - where you can't just import the exact same type definitions, etc.

This also sounds like the Nix language (https://nixos.org/).

Have you seen aws-cdk or pulumi? Does exactly that with TypeScript to boot :)

Yep. I prefer to use language files over JSON or YAML files if I can.

For aws, I usually read the cloudformation/api/boto3 docs. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

I don't know all the options, and I don't need to either. Can look at examples to get started and customise as I need. What changes?

Idk if other ways are better, just looking at large text config files for apache or other linux software boggles my mind just as much if not more. Once you have lots of options it's just going to be complex to understand.

I think clouds have been trying to fool developers into thinking that cloud is easy. Don't feel bad, building a reliable infrastructure is incredibly hard. There're ton of services, some of them are redundant, and it takes a lot of time to discover all the nitty-gritty.

Maybe it's messaging intended to be for grizzled old veterans going "look how we've taken the old complicated thing and given you powerful tools to allow you to manage it more easily with less steps" but then gets interpreted by neophytes as "look how easy it is for a complete beginner to come in and set up a complicated system".

Great observation. A lot of messaging for cloud products is comparing against legacy or dominant solutions. A beginner doesn’t have that reference point.

Messaging is written by marketing and sales departments.

Employees there usually don't have depth of technical expertise or experience to draw from, and are incentivized to make the product sound as easy to use as possible.

This combination results in {really complicated thing} -> {simple abstraction}. Ultimately, everyone internalizes this and starts forgetting that {simple abstraction} is actually {really complicated thing}.

Happens with any popular technology. Networking, DB, cloud, ML.

Unfortunately, this flies because the primary buyers are also non-technical.

>I feel increasingly dumb the more I see these configs.

Especially since there is no IDE support.

I've come back to enjoy XML with nice schemas and IDE validation and autocompletion instead of YAML and liquibase changes not going through properly due to an indention problem.

You can use schemas with json/yaml (e.g. jsonschema or protobufs); I wish people used them more though

Urgh, yes.

I'm not a dev by any stretch of the imagination but deploying PHP apps on servers was significantly easier than this mess we have now.

Rsync/ln is still the fastest deployment process I’ve had so far.

Still too complex for me, let me just login to an sFTP and drop some files in

And get the second order effect of one of those configuration options wrong and wake up to a huge AWS bill. And, yes, with a bit of groveling they'll probably refund you after you've sweated a bit--the first time at least.

Or worse, you fudge a YAML line and bring the application down.

There are tools to catch and prevent that sort of thing: Datree, AWS CloudFormation templates, and Sentinel, just to name a few. They apply and enforce different policies to prevent YAML/JSON (or “infrastructure as code”) misconfigurations.

(Disclaimer: I work with Datree, but not the others.)

> yml/json configurations for [...]

We learned this lesson the hard way with XML back in the early naughts when behind everything was a nauseating morass of xml with overcomplicated designed-by-committee schemas.

People moved out of development and tried hard to forget it, but neglected to instruct the next generation about the pitfalls of shitty configuration files that get out of control.

Now we got json and yaml. No more </>'s, yay, but somehow worse, less maintainable and less understandable.

No, I'm not overwhelmed by documented/structured config files.

20 years ago I was a Windows admin, discovered Linux, and it blew my mind how much easier it was to configure, experiment, validate, and roll-back config changes. GUIs are nice for discovery and exploration, but there's a very good reason they're not used for server and infrastructure configuration.

There's definitely room for config-viewer plugins in VSCode, Sublime, etc. to help write and maintain the 100's of config files we have to be knowledgeable about in DevOps these days.

Does anyone have good suggestions in this space? Or do I just have to google docs and reference documentation when I need to figure out what the hell all these parameters mean?

Track changes in version control (git) using infra as code tooling

The new J2EE.

Reply on news.ycombinator.com