You should validate that the request is signed correctly, and reject it if not. An IP ACL can be a useful "remove some of the noise" tool, but no more than that.
For example, Discord's webhook implementation has you verify the incoming request against an app-specific (your app; not all apps) public key: https://discord.com/developers/docs/interactions/receiving-a... — this is really the right approach to this.