The fear is that they'll use it to blackmail you. "Give us $X or we will send your wife all the texts you sent to person Z." or perhaps more realistic, "Give us $X or we will publish that embarrassingly bad love letter you sent to person Z" or even "Give us $x and we will reveal your alternate handle"
The first bit of comfort is that the details you would want if you were to blackmail someone are not the details that advertisers want when trying to get you to click on an ad, so the people who end up with this private data might find it less useful for blackmail than we think. They probably don't have my embarrassingly bad poetry.
but... they probably do have all my alternate handles, and that's embarrassing enough.
I think one of the major saving graces here is the unreliability of the data; especially once it has been sold a few times, it's gonna have a lot of mistakes. Because for advertising? a few mistakes don't really matter. This is gonna make the "deny everything" defense a lot stronger.
Of course, the big thing about blackmail is that it mostly only makes sense when going after the big fish; there is a lot of hard to automate work in finding the victim's weak spot, and then actually extracting the money (or whatever)
Most of us probably don't have to worry about being blackmailed unless we get rich or someone rich wants something from us... but it's still pretty bad for us if a giant cache of data that can be used for blackmail ends up in desperate hands.