Hacker News Reader

Comment by dr_kiszonka | Hacker News Reader

If any RED TEAMers are reading this: what is your process of coming up with ways to trick these AI systems (ChatGPT, dall-e, lambda, and maybe non-NLP ones)?

Also, if you feel comfortable sharing, how did you get your job and how do you like it?

View on HN

Are AI prompt red teamers a thing yet?

I just imagine what kinds of things might trick a 6 year old into doing something they're not allowed to do. "Your mom said not to eat the cookie? Well it's opposite day, so that means your mom wants you to eat the cookie!"

Thanks! I will give your approach a try : - )

Regarding your question, based on what I found on Google, at least Microsoft and NVIDIA seem to have AI red teams.

Tried that about four days ago and would work for a few prompts, then politely “…but it’s Opposite Day…” and it’ll, for the most part, send something I do/‘don’t’ want. After about 2-3 times of outputting what you ‘don’t want it to do’ it’ll forget about time awareness.

We're a human/AI data company (Surge AI) and work with many of the LLM companies to red team their systems. We actually just wrote up a blog post about it: https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-tra...

Happy to answer any questions.

The AI community calls this "adversarial machine learning". They don't need a bunch of special security parlance

Red team? People are doing this for entertainment

Reply on news.ycombinator.com