Hacker News Reader

Comment by ozr | Hacker News Reader

It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.

View on HN

Sure, these particular examples are concerned with morality, but the problem is more general and limits the value of language models because they can be hacked for other purposes. A good example that's been going around is having an agential model that manages you emails. Someone sends you an email using prompt injection to compel the agent to delete all your emails. Or forward all you emails to another address.

If there isn't a way to secure the behaviour of AI models against reliable exploits then the utility of the models is dramatically limited.

Use multiple models or gap their capabilities.

Don't allow it to delete emails (maybe allow mark for delete in 30 days?) and have a whitelist of acceptable forwarding addresses or push a confirm/deny notification to a manual reviewer.

It's like AI diagnosis, we aren't going to run it full stop automated without safeguards on top or manual review for a long time.

Is the issue you have with the group of people doing the moderation, or with the idea of the moderation in the first place? Are you certain that its the 'SV engineers' that are doing the current moderation? If you think the problem is with the current group of moderators, who do you think should be moderating and what should be the criteria of their moderation? If you think we don't need any moderation, do you believe that people should have a fairly easy access to

  * Learn how to make bombs (as mentioned in the article)
  * Get away with committing crimes?

Are moderating these topics related to morality?

It’s not exactly difficult to find the resources from which the LLMs probably learned these answers in the first place.

I can think of many things an LLM could do that would be far more harmful than any of this.

The Progressive laid out how to build a hydrogen bomb: https://progressive.org/magazine/november-1979-issue/

The US government said that info was born secret and sued: https://en.wikipedia.org/wiki/United_States_v._Progressive,_....

I won’t spoil who won the argument.

When the "crimes" in question are e.g. drug use or abortion, yes, moderating these topics is very much related to morality.

In the United States, I can write a pamphlet about getting away with crimes and making bombs and hand it out on the street. There is nothing inherently illegal about those topics.

Just don't talk about jury nullification[0]

[0]https://www.usnews.com/news/articles/2017-06-02/jury-convict...

That conviction was overturned.

https://fija.org/news-events/2020/july/keith-wood-conviction...

Huh, well good to know. Though not before his life was upturned and he did spend two weekends in jail.

Indeed. Why do they get to decide for humanity?

Because they're private corporations and these models are their private property, indeed they are a type of capital like a bottling machine or a loom.

This isn't so much about moralizing as much as it is businesses deciding what to do to make the most money. That doesn't mean you can't disagree with it, far from it. But I think the framing of, "these companies are imposing their morality on me" is a misdiagnosis. I don't think it's really a moral position for them, it's a product engineering position.

I would describe the situation as, "more and more of the world is controlled by large corporations, and I'm increasingly subject to their arbitrary and unaccountable decisions. Many of which make no sense from my vantage point."

They are proving that the present techniques being used to control their models can be reliably bypassed. Regardless of what you want your model to be able to do, there may be things that you don't want it to do. For instance, if you are making a product for children you'd probably want it to avoid certain content.

If you are training you own model, it would be nice to know what, if any, techniques you could employ to balance the effectiveness of it with generality.

Reply on news.ycombinator.com