Hacker News Reader

Comment by buildbot | Hacker News Reader

Disclaimer - I work at Microsoft

They put a lot of work into the research to make this safe already - they had the model red team itself, in a reinforcement learning environment probably. It won't be perfect, but at this point, they need a huge user base to adversarial test it any further. Otherwise no progress at all would be made.

edit - I should say I have no internal knowledge about this.

View on HN

And that research is valuable, but it's definitely nowhere near the point where safety/alignment progress can't be made without a user base. A few recent favorites of mine: https://arxiv.org/pdf/2212.03827.pdf https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solid...

Note that that kind of research was only possible on the (smaller) open models. When the new models are proprietary ones then outsiders can't do that kind of analysis. If they had regenerated the tokenization dictionary to match the training data instead of reusing an older set then this wouldn't transfer from GPT2 to ChatGPT.

Safety is a subjective quality. What are the downside of "put"[ing] a lot of work into the research to make this safe already "?

Do you think MSFT has rolled this out too soon rather than getting it tested with huge test user base like their own employees similarly to what Google is doing?

No, I think to really get these battle tested it needs to be in everyone's hands. Microsoft is a limited selection of people, and the world is vast. Plus, internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.

>> internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example.

>> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.

Hopefully better than previous attempts to put AI on the Internet:

https://youtu.be/ajGX7odA87k?t=20m12s

Reply on news.ycombinator.com