I've taken a close look at AutoMQ. By offloading data to EBS and S3, it turns Kafka Brokers stateless, avoiding cloud provider traffic costs and shifting the architectural complexity to the cloud provider. It's really killing two birds with one stone.
Thanks to the AutoMQ team for their excellent work. I'll be giving AutoMQ a try in an upcoming project.
The biggest challenge I face with Kafka is constantly having to migrate data. Do you experience this too? I'm hopeful that a shared storage architecture will completely fix this frustrating issue.
Cool. Lower latency, lower cloud costs, and lower operational costs — I love this product.