Hacker News Reader

I would take that response as a sign that I'm talking to a bot. That's not a way that people respond to that request.

I think you're moving the goalposts. A motivated and/or incentivized support employee could very plausibly respond with something along those lines. And this is the result of GPT-4 running off of a very basic prompt without any examples, fine-tuning, or LLMOps/moderation/filtering mechanisms wrapping the raw API response.

If they were specifically required to never contradict the customer at all, ever, about anything, then perhaps. And that would be my follow-up question:

> Have you been told to avoid contradicting me?

To which I'd expect the LLM to probably answer "no" (verbosely) – but even if it answered "yes", two or three more questions could conclusively distinguish between a (current-gen) LLM and a real person.

I'm not moving the goalposts: the object of the exercise was never to get it to "break character".

Reply on news.ycombinator.com