FWIW, you can get Kafka packaged as a fully managed and HA service from https://aiven.io on AWS and also Azure, GCE and DigitalOcean.
But if the Auth0 runs their entire operations on AWS, maybe Kinesis would have been a more natural transition.
But if the Auth0 runs their entire operations on AWS, maybe Kinesis would have been a more natural transition.
To anyone reading this with Kafka experience, do you have any tips/advice when it comes to maintaining a Kafka service?
Use configuration management such as chef to allow you to quickly build new nodes and to roll out changes accross the cluster. You will need to make tweaks. The chef Kafka cookbook which is the top result on Google has means of coordinating restarts of brokers accross the cluster. Use consul as a locking mechanism for this. You could use zookeeper, but consul works well for auto DNS registration and auto discovery.
Use the yahoo Kafka-manager app to manage the cluster and to see what is going on.
Don't use the Kafka default of storing data in /tmp/. Your OS will periodically clean it.
That seems like MySQL level of bad defaults.
Thanks for the tips.
If you want highly available ZK, your choices are 3, 5, 7... nodes, for which you can have 1, 2, or 3 nodes offline at any one time.
If you have one node fully down on a 3 node cluster, and there is even a tiny network blip or partition (as often happens in cloud environments) then you are down.