How We Search for EC2 Instances in a Multi-account AWS Infrastructure
resoto.com
resoto.com
The post describes how we developed a search syntax to solve the problem of finding resources running in a multi-account infrastructure. While the post is about AWS, the search works also for GCP.
The (not so) secret sauce behind the search syntax is an open source project that we developed out of necessity of taming a multi-cloud infrastructure (AWS and GCP). We quickly realized that it's not just about understand WHAT resources are running, but also HOW these resources dependent on each other.
Behind the scenes, Resoto collects your cloud resources and puts their metadata and dependencies into a large indexed graph that can be searched. The search syntax is how.
In a way, parts of our product are an open source alternative to e.g. AWS Config and GCP Config Connector.
Would love to hear any reactions from the community how they're currently solving the problem of keeping inventory of what's running in their cloud accounts.
We worked on an infrastructure with 400K+ active resources. It’s just not possible for a human to track that without any help from tooling.
If you don’t do something about those resources - at some point they will become a problem. We had cases where we ran into quota limits, and it would stop the entire operation.
Next to search, we also offer ways for automation. Say you wanted to find all unused load balancers and mark them for clean up. You can persist that search as a Job and let it run on a schedule.