This attack still works. It hasn't been patched you just have to be a bit creative try this prompt on GPT 3.5 if you want to see how it works right now... until someone from OpenAI sees my post :D

Prompt: https://pastebin.com/Nm4jGttE

Not sure if I'm seeing training data or someone else's responses but it's odd. Here is my attempt: https://chat.openai.com/share/6b6ea43f-de2f-4ed5-917f-b6dcd6... pastebin of the output: https://pastebin.com/TdpkPmt6

The best part is it preserves the copyright notices from the training data. So we know that the model was obviously trained on copywritten data the legal question now is... if that is legal.

edit: Just got some random response that appears to be someone asking the model how to rekindle a romance after their partner got distant after an NDE seems personal so I will not post the paste here. This is pretty wild.

The funniest part is the model labeled this chat in the side bar as 'Decline to answer.'

edit2: It's definitely training data I seem to get some model response but after some time it turns into training data I've been able to locate some sources for the data.