It is an interesting feature request, I create an issue that you can follow the progress:

https://github.com/airtai/faststream/issues/758

Not natively, but you have a tools to run multiple brokers manually (run them together at lifespan hooks). In the nearest future we will add this feature at the framework core.