their recent blog post doesn't sound that far off from the OP's architecture: https://www.confluent.io/blog/cloud-native-data-streaming-ka... (discussed: https://news.ycombinator.com/item?id=36870278 )