Ask HN: How can I overcome anxiety over dealing with infrastructure? | Hacker News Reader

Ask HN: How can I overcome anxiety over dealing with infrastructure?

15 pointswatermelon594 comments

Not kidding: having to change - or even worse, add - anything to my team's Terraform files gives me real, serious anxiety.

As someone who's been strictly a developer for all of my 15-year career, infrastructure just doesn't "click" with me. It's a bunch of arbitrary concepts each with tons of arbitrary properties. There's no cohesive logic to it, and I find it impossible to form a mental model of those things.

I'm currently responsible for creating a new service, and because I have to add a lot of new Terraform definitions to those files, I've literally been avoiding work these last few days. I can't reason about it. Even if I write something, it's annoying me to no end that I can't look at it and analyze whether it's complete and sound. All I can do is trial and error.

I understand individual concepts (like what AWS ECS and ECR, or Elasticache, or whatever), but the "glue" looks so inscrutable. Sometimes two things are connected together by a "chain" of definitions, but it's not really a chain because it's really hard to read those files and understand the association between components.

Does anyone else suffer from the same problem? How can I get better at this?

Part of it is the multiple layers of abstraction. You aren't really sure what is going on when you run terraform apply.

I recommend turning the TF_LOG environment variable all the way up so you get super verbose logging. Then you can actually see what terraform is doing and it actually isn't too bad. It makes a bunch of calls to AWS one by one.

My guess is you just haven't worked with infrastructure that much so it is familiarity problem. You are also using a tool that has a very high level of abstraction. So getting more familiar with how infrastructure is setup (even manually) will help.

Some other things:

- Visualizing things by drawing things out can help - Keep the AWS web console open and you can see Terraform creating the resources on the fly by refreshing

Here's a view from a higher level...I spent 25 years looking after network infrastrucutures...infrastructure uses up a lot of headspace. The work is never finished and the alerts never stop. If you are good in the space you are shown respect as others don't usually understand what you do. However you are looking after something which people just expect to work.

What helped me was doing something totally different at the weekend. It's no accident I re-landscaped my back garden by hand and built two sheds. You will never get a light breeze blowing in your face, the sound of rustling leaves, in infrastructure. Doing something different made me happy.

Don't wreck your brain either trying to understand everything. Become good at what you need to know. Technolgy is awesome but there are paradoxes. When I was in datacentres I used to think about the great damage I could do with a pair of children's scissors on fiber optic cables. This somehow brought all the awesomeness of the modern world down to a level I could undertand :-)

> "glue" looks so inscrutable. Sometimes two things are connected together by a "chain" of definitions, but it's not really a chain because it's really hard to read those files and understand the association between components.

Give us some specific examples that you have figured out already and some things you are trying to figure out. Be technical and specific.

Is all your infrastructure in code? Are you having trouble following the code? Please elaborate and post some examples of things you have figured out and things you have yet to figure out.

What questions do you have? Brain dump them into this post.

Focus on getting a solid BTD (build test debug) loop first. It doesn't have to be fast (especially for infra), but it does have to be deterministic. Try to start small. What's the smallest TF file? What does TF really do? (Another poster mentioned sending commands to AWS).

You could also not do infrastructure if you don't like it!

Reply on news.ycombinator.com