This honestly sounds like a job for the BEAM (Erlang/Elixir).
The preemptive scheduler would cover almost every need mentioned in the article.
The preemptive scheduler would cover almost every need mentioned in the article.
It does have some drawbacks though: if you have thousands of mirrored or quorum queues, managing the cluster becomes more cumbersome - it can take a while to replace nodes (rebalancing thousands of queues around). Also, the management tools (the web UI especially) don't perform that well if you have 10,000s of queues and 100,000s of consumers
The fairness of the BEAM scheduler is not the same as multi-tenant fairness. I'm aware of lcnt in Erlang that helps with contention, but that will have a hit in throughput like any other locks.
Unless I'm missing something?
If you wanted to make it truly fair you could spin up a GenServer per tenant and you would have a max concurrency per tenant of 1, but still all executing equally in parallel without 1 tenant stealing CPU time from the others. Contention is a non-issue. Fairness is built in.
You get the fault tolerance, isolation and observability as a by product too.
That's just my first reaction on reading it.