Ask HN: Maintain zero knowledge while using IP logs for attack mitigation

I've got a service I'd like to put out for public use, but I need to protect it from attacks. Typically I'd use fail2ban and read the IPs from the logs for rate limiting, banning, etc.

Is there a way I can one way hash IPs in such a way that if requested, I could honestly say "I have no way to give that to you"? (HTTP and some UDP requests)