When I last checked my google ad profile (and this was a while ago; now it contains a lot less info probably because I opted out of a bunch of stuff), it was full of errors. Google did get my gender right but they put me in the wrong age range and a bunch of interests were out of whack. Good thing this was for ads, rather than a heuristic to ban me.

Best of the best, top of the class is not infallible. It's good enough to sell, because even something as low as 80% accuracy is good enough for things like "Do you want to subscribe to weekly pop tv news", it's not good enough when at the other end you get your account banned.

Google has, on this very site, built up a horrible reputation of using automated processes and having too many users to give those affected by those processes some good recovery. What you're describing is a recipe to replicate that.

99% is not good enough. 99.99% accuracy gets you 0.01% false positives. That's crazy low, right? It's also 100k users when you have 1bn users. Those fancy AI processes are nowhere near that.