That's a slippery slope; if my neighbors plant tulips and I HATE tulips, I don't get to sue for damages, even if the tulips weren't there when I moved in.
Legally, that isn't even remotely a slippery slope. There is a vast difference between a decorative object located solely within one's property which has no effect on neighboring properties, and an for-profit unlicensed/illegal activity which affects the neighbor's safety and use of their abode.
Yeah, no. That's not a slippery slope. That you just being obstinate.