Hacker News Reader

Comment by celeritascelery | Hacker News Reader

From the Copilot FAQ:

> The technical preview includes filters to block offensive words

And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

View on HN

In my testing of Copilot, the content filters only work on input, not output.

Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.

Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.

I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.

I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious.

https://spectrum.ieee.org/tech-talk/artificial-intelligence/...

The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.

Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay.

[1] https://docs.github.com/en/github/copilot/about-github-copil...

Maybe, but even if so, filtering the output would also prevent this.

Easily offended AI is exactly what the world needs

We have too many easily offended NPC's as is.

Yeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG.

eyeroll

They probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already

Interesting how this continues to be an issue for GPT3 based projects.

A similar thing is happening in AI Dungeon, where certain words and phrases are banned to the point of suspending a users account if used a certain amount of times, yet they will happily output them when it is generated by GPT3 itself, and then punish the user if they fail to remove the offending pieces of text before continuing.

Lol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.

GitHub is owned by Microsoft, which has some relevant experience with publicly accessible AI (Tay): https://en.m.wikipedia.org/wiki/Tay_(bot)

Maybe Github just doesn't have many repos to control death factories and execution squads?

does it also censor "lesbian"?

Blocks offensive words, but doesn't block carefully crafted malware.

Reply on news.ycombinator.com