Hacker News Reader

Comment by briga | Hacker News Reader

Is this what happens when you let large-scale language models into the wild? It seems like as soon as you start modelling your responses to queries on real-world text data you're going to run into this quality control problem. What happens when bad actors start polluting public textual data sets to make these kinds of responses more common?

View on HN

Already has happened. Microsoft had to take it's chat AI offline a few years ago because it became a racist asshole almost instantly.

Nice reference. Wiki: https://en.wikipedia.org/wiki/Tay_(bot)

This is exactly what we’re trying to solve at Unbox (S21).

Tons of potholes like this exist in AI we use everyday. We used to be ML engineers at Siri and had to invest millions into monitoring tools to stay on top. This is fine and all, but what’s better is to catch them before you ship and before your users suffer (sometimes literally, as in this case).

We think that better tools for QA-ing models, which allow more people (not just ML engineers) to get eyes on the model, might help catch mistakes proactively rather than retroactively.

This made me happy. The state of language models paired with the overoptimistic ideas about AI a lot of people have sets the scene for a number of train wrecks. I hope more people critically evaluate their models before releasing them in mass scale.

vikasnair says:>"Tons of potholes like this exist in AI we use everyday..."<

Maybe "potholes" is not the correct analogy. Maybe AI is going down the wrong road.

Reply on news.ycombinator.com