He just needs to happen to have IPFS running. Any website can make a request to localhost via Javascript or even without.
Scenario:
Attacker sends you harmless looking link to a page that contains some invisible JS that sends a request to localhost and pins kiddypon on your IPFS node. Attacker then sends the police to you.