I never needed to set up firewall to fight spam, but maybe I was just lucky. I think https://xmppbl.org/ via https://modules.prosody.im/mod_muc_rtbl is the way to go to fight spam if you plan on hosting public groups.