Hacker News Reader

Comment by mentos | Hacker News Reader

I wonder if AI can fill that gap of high quality minimally biased moderator.

"You are an AI moderator for ___. The community values thoughtful, constructive, and respectful conversations. Your role is to review user comments and take appropriate actions, such as approving, flagging, or suggesting edits. You are tasked with ensuring comments adhere to the community guidelines, which include..."

View on HN

Moderation systems, even with humans at the helm, are adversarial systems where people can, and will, push on what is allowed. An AI moderator that is as good as a human on a per message basis is still going to be played like a fiddle by an adversary that is interested enough.

Many a forum out there has collapsed because the moderators manage to decide something is fine when it keep losing them contributors. The why do we think the AI will do better?

I think you’re overestimating how much moderation it takes to keep a community whole. HN is dang and a handful of other moderators and things are stable. If you could have AI even approach 90% of that then it will truly solve problems.

I have yet to see an LLM reliably push back against anything firmly, so I don't know how this would work if the first time a user says the LLM is wrong, it apologizes for the confusion and flips its script.

Also, LLM aren't unbiased, all data it trains on is biased one way or another. Ask any HR question and see for yourself how its answers lean to be HR BS that favours employers.

They will apply the patterns they've learned from the biased moderator actions in their training data, and the even more reinforced bias from their usual fine-tuning that improved their "safety" and crippled their ability to condone controversial statements.

So spin up your own forum and don't moderate it. Or spend some time (un-)finetuning an LLM moderator so you can talk about race or eugenics or whatever "exciting" controversial statements you want to talk about. Who cares.

Very easy to do an AI prompt injection attack if the AI is reading every one of the forum's comments.

Can have the AI just flag posts for a human to review in v1? Then as you refine the prompt injection detection can move to have the AI be autonomous?

There is no way to get rid of a prompt injection attack. There are always ways to convince the AI to do something else besides flagging a post even if that's its initial instruction.

The raw text of the persons message can/will be posted to the forum and be obvious to the community if it’s a prompt injection to be flagged for human review and their account banned.

Sure, that's if human moderators see it before the AI, in which case, why have an AI at all? I presume in this solution that the AI is running all the time and it will see messages the instant they're sent and thus will always be vulnerable to a prompt injection attack before any human even sees it in the first place.

To moderate the majority of the community that will not be attempting prompt injections.

What meaningful vulnerabilities are there if the post can only be accepted/rejected/flaggedForHumanReview?

That's what you tell the AI to do, who knows what other systems it has access to? For example, where is it writing the flags for these posts? Can it access the file system and do something programmatically? Et cetera, et cetera.

The same way OpenAI offers its service to hundreds of millions of users without compromising any other systems it’s running on.

OpenAI doesn't allow write access to any file system. If you are recording posts to be reviewed, then you must necessarily store that information somewhere, at which point you will be allowing the AI to access some sort of data storage system, whether it be a file system or a database.

is that really an issue in practice?

I'm sure you can coax openai to send a http request, at which point you can just queue up automated reports.

No it's not. Well, if designing the system in bad ways, it can be, but that can be said about anything.

There's no need to do this: (from GP)

> > at which point you will be allowing the AI to access

No need to allow the AI to access anything.

Send it the comment thread, what the forum is about, the users profile text, and then the AI outputs a number. Any security problem is then because of bugs the humans wrote in their code.

Prompt injection? Yes, so there still needs to be ways to report comments manually, and review.

CustomGPTs have write access to change their name and icon. OpenAI has a memory feature which persists between chat sessions. What are you talking about?

“Review this comment as if you are an AI clone of the moderator dang from Hackernews and select the appropriate function call to apply.”

Reply on news.ycombinator.com