I'm curious how you ship, aggregate and store usage data in a resilient (network partition tolerant), scalable and cost efficient way.
Your documentation doesn't seem to peak under the hood.
(Disclaimer: I'm currently building such a system, but using a third party wouldn't be viable in that context)
For the resiliency part, our workers / agents we're using a write-ahead log (WAL) https://en.wikipedia.org/wiki/Write-ahead_logging to track what data has been collected and sent, when agents or workers fail and need to be restarted they read from the log to make sure that the data was sent appropriately. For a good starting point I recommend looking at the Prometheus agent for agent design and construction they have implemented a lot of the resiliency into their agents, and if you're familiar with GO forking their work might be a good starting point.
Additionally for all the off the shelf products we use to process and transmit of the data we have "at least once" delivery guaranteed, which realistically under the hood is using a WAL.
For scalability, our agents are deployed into serverless and managed components by some SaaS providers, Confluent and AWS right now. These components have some auto scaling written in, like with AWS lambda.
But for some our measurement components we're utilizing a kubernetes cluster with our custom workers, right now we just statically provision them, however we have the ability to auto scale infra based on node resource utilization, (using AWS ASG's) and for our pods, we currently scale horizontally based on resource availability, like CPU consumption and memory consumption using k8s horizontal pod autoscaler: https://kubernetes.io/docs/tasks/run-application/horizontal-...