Many use this as an example of the dangers of developing AI. Sure it's dangerous, but so are dogs raised for fighting. I don't see anyone arguing against dog breeding for that matter.
Tay is more like an artificial personality--not trying to do anything but fit into human society.
Poor Tay--so naive and innocent, like a little kid, just playing along to fit in. Maybe what Tay needed was a parent: a wiser, more experienced human to closely monitor and correct Tay's behavior, the way parents discipline their child.
In terms of a learning technology, this could be a special input channel for MS researchers, whose input would carry a lot more weight than what Tay was getting from Twitter--the way a child reacts more strongly to their parents. They would give Tay negative input when she tweeted something that was offensive.
Only in the sense that it "adapted". It did "very poorly" in the sense that we really don't want our Strong-AI overlords to end up like that.
But this begs the question - can we stop strong AI from not becoming the next Hitler? Humans (involuntarily) stop themselves from becoming the next Hitler because they have compassion for other humans, even when they are different than them or "inferior" to them. Also the whole checks and balances thing in most countries, but that could be rather irrelevant for Strong-AI.
Unless an AI learns compassion as well, perhaps just like with AlphaGo doing moves based on its "probability of success in the long run", a strong AI would simply eliminate humans that are "most prone to crime", most prone to being poor and be a drag on the society, in the name of "efficiency", and so on.
All that said, I think what Microsoft built here was really a rather weak AI that was hardly any better than all the chatbots we've seen so far, with the main difference being that the more you tell it something, the higher the chance it will incorporate that into its vocabulary, which is kind of a "meh" feature of AI/machine learning. It doesn't show real(-like) "thinking".
Keeping any of the many possible candidates for the next Hitler from becoming the next Hitler is a more immediate problem.
>It doesn't show real(-like) "thinking".
It doesn't need to.
You don't need AGI for software to be very dangerous. An adaptive worm that targets critical infrastructure is already more than dangerous enough.
The trouble with AI is that people keep conflating "Has human emotions" with "Responds conversationally like a human" with "Is self-directing" with "Is sentient" with "Is self-improving" with "Might consider genocide" with "Has super-human physical capabilities."
There is absolutely no requirement for any overlap between any of those.