Hacker News Reader

You want to be careful of async inserts as it's in-memory buffered as well and is not replicated afaik. So you can lose a bit of durability when using it (or you make sure your upstream pipeline can retry).

It is really nice in a pinch though.

View on HN

My read on it was that if you keep wait_for_async_insert=True then the client will still wait until there's confirmation of a successful transaction, giving the possibility to retry. It does require one to actually have a retry handler on the client side, but that's a good idea anyways.

In practice, using async insert with wait means you need batching/retries implemented on the client side or else you may find you have thousands of threads stuck in wait. If you need to replay bc you messed something up you're also SOL. I generally appreciate not having to run kafka but that comes at a cost.

Very good point.

Note that you will need a thread pool that’s large enough to handle 1 second (or whatever flush timeout you set) of events in your app concurrently. If you have thousands of events per second, plan accordingly.

Reply on news.ycombinator.com