I agree - it's a problem worth solving.
Re: not having data to QA model, you can use the distribution of the generated labels to inform model updates. Alternatively you can get labels at the edge using a local edge interface then use those.
Re: not having data to QA model, you can use the distribution of the generated labels to inform model updates. Alternatively you can get labels at the edge using a local edge interface then use those.
1. Model development, improvements and tuning (choosing correct preprocessing, features, architecture, introspection, debugging…). Client-side unsuitable.
2. Training / updating of fixed, well-designed models on new "unseen-by-human-but-otherwise-well-behaved" data. Client-side suitable.
So far, the former has always been the bigger challenge, hence my QA concerns.
But perhaps once your problem is well-understood, and data known to be equally-distributed and well-behaved, the latter becomes increasingly useful. It's definitely an exciting new direction. The OP identifies personal / PII data as the flagship use-case, which sounds about right.