As someone who's made a career out of managing Facebook ads, I can understand the fear surrounding them.

I've only ever use targeting/tracking to serve users content that is super relevant to them. And this is reflected by there clearly being far higher engagement on FB event-targeted ads vs general ads.

Facebook has been extremely strict on the types of ad copy and creative we can share with users via ads. I am a very ethically consicuous person and not once have felt that I've invaded anyone's privacy with the ads. The targeting parameters aren't as powerful as to allow targeting more granularly than several hundred users at once.