Don’t share it publicly with no viewing restrictions, yes. If you want to limit who can view it, then limit who can view it.
It’s like driving around with a banner on your car and then complaining because a certain subset of the population looked at it.
There are many ways to restrict access to content. The simplest is to create a login and TOS that restricts usage. Just do that and it won’t be used for any training.
But putting it on a server with anonymous http access means that the whole world (including robots) can see it.
It’s not victim blaming, it’s just reason. If you don’t want people to see something, don’t show it to them.