There's something to be said for not shutting down someone unless they break a law. I do acknowledge the court would need to be much faster and easier to access for these kinds of things though, you don't want to burden a provider with enormous legal bills just to deal with this kind of thing.
If I host a service for manchester city fans, why would I not be able to ban annoying united fans? Who cares if what they do is legal. What if I post NSFW content here on hn. It's not illegal so it's okay?
Just this month content creators from Russia or belarus have been banned from dozen of services, they did not violate any law.
It's ridiculous to think that yout platform isn't free to apply it's own rules.
We're talking about platforms that are not so restrictive in their topics.
Unless you can prove in a court of law you're being restricted for illegal reasons, there's not much you can do.
Internet is full of stories of people losing their livehood overnight in these services and there isn't much one can do.
How? You can't preemptively determine the output of a Turing-complete program without solving the halting problem, so while Google will always try to do their damnedest to predict a bad actor before they get to the store, they will definitely always fail to find some.
If they can't ban them when they find them, what should the alternative be?
This feels more like handwaving than brainstorming ;-)
It's a tricky problem and I think your suggestions should try and reflect that as much as possible.
You obviously can't eliminate all abuse but it can be managed to an acceptable degree with thoughtful design.
This is getting away from my larger point that the incentives here are screwed up. Why is that? Why doesn't Google fix their shit? What is blocking this from fixing itself? Usually there is an incentive like threat of lawsuit or people leaving the platform.
Yes, perhaps we could come up with a set of rules that users must abide by to use the service, such as not being abusive. And if they're found to be breaking these rules, we could then restrict their access to the service. The worst actors would have their access restricted permanently.
If you're suggesting "stop 100% of all abusive content before it happens," that's just silly.