There is a whole KIP that is “preview” on Kafka 4.1 to handle this use case natively: https://cwiki.apache.org/confluence/plugins/servlet/mobile?c...

Note: I haven’t had a chance to test it out in anger personally yet.