One aspect that concerns me even more is the profiling and labeling of individuals.
I have a friend who is - as he puts it himself - "even beyond the [autism] spectrum". He recently showed me how often he's falsely accused on Reddit and Discord of being a bot or of having used ChatGTP due to the way he writes. It happens so much and in such aggressive tones that I'm surprised he's still bothering to engage with others on the internet at all. Very sad to see.
It wouldn't surprise me if this AI would pick up on his strange writing style (hard to describe.. very structured) and decide his reviews or whatever consistently aren't genuine or human and brand them accordingly. He wouldn't even be aware of the fact that his Amazon reviews are being invalidated and pointless to spend any effort on since he doesn't use Firefox himself.
There are only two things Mozilla could do to remedy this kind of situation: have their AI also track and profile people with possible mental/neurodevelopmental disorders, or have a system where people themselves can prove to Mozilla that they're human and can't help writing in that manner due to the way their brain is wired. Neither of which I am sure we can all agree on would acceptable in the slightest.
This is a really messed up development and it's been making my friend really anxious about what other things will be coming in the future.