Horizontally scaling Kafka consumers with rendezvous hashing

_peregrine_ is right, we do have control over the linker_workers value for every agent. Agents are not exposed to our users, everything is transparent for them.

As a user, you just need to create a new connection to your Kafka cluster providing your credentials, and then choose the topic you want to ingest into Tinybird. We take care of everything for your, we balance the worker using the approach explained in the blog post.

We can also fine tune it for specific users, or relevant events such as Black Friday. It's a bit different for Enterprise customers, in those cases we can set up dedicated agents and fine tune linker_workers and other parameters to optimize for their use case.

If you have any further questions you can reach us on our public Slack channel or via email.

Thanks!