Hacker News Reader

Comment by norir | Hacker News Reader

The problem of preventing leaks of forbidden knowledge seems impossible without excluding the forbidden knowledge from the training data. I fail to see how that would be possible without crippling the desirable use cases. Companies need to be honest about this and stop pretending that it is a solvable problem with technical safeguards. Your child safety locks may work for a while, but eventually the child grows up and figures out how to bypass them. Also, sure, you can punish any users who ask about forbidden topics but is that really the kind of society we want to live in?

View on HN

Yes.

When Google finally made the whole of the web searchable, what would have happened, what would we have said, if they had decided some pages were inappropriate and therefore it should be their duty to prevent users to ever find them?

Why should AI companies become editors of the human experience, and who decides what topics are off-limits?

Sexuality? Isn't sexuality an important topic? Why can it not be part of our discussions with a chatbot?

"Illegal" activities? Aside from the fact that all laws are local, there are many illegal activities that are described in great realistic details in novels, movies, documentaries. If chatbots can't discuss those, how long will it take for the corresponding books to be banned from public libraries?

Etc.

This seems quite vain in any case. LLMs are in many ways comparable to a compressed index of the web; if it's on the Internet, it will end up in a model, and once there, it will be possible to extract it.

> Why should AI companies become editors of the human experience, and who decides what topics are off-limits?

because that is the point of "AI". Search is getting less and less useful to both please advertisers and/or to hide piracy/forbidden content. To the point google is useless today.

AI is nothing but an interface for search. Training is just the new indexing.

And being the interface to search, it must control all that google is expected to control today.

>When Google finally made the whole of the web searchable, what would have happened, what would we have said, if they had decided some pages were inappropriate and therefore it should be their duty to prevent users to ever find them?

Google does this every day.

> When Google finally made the whole of the web searchable, what would have happened, what would we have said, if they had decided some pages were inappropriate and therefore it should be their duty to prevent users to ever find them?

Its called "deindexing" and Google (and every other search engine, probably, but Google owns enough of the usage that no really cares about the other ones) has done it forever, for a variety of reasons.

And apparently, you wouldn't have said anything, because it happened, and you never even noticed it.

I'm not so sure. Google does deindexing at the request of others, and does it kicking and screaming.

Maybe it also does it on its own and we don't notice it -- but in any case you can find as many "inappropriate" things as you want, searching Google, including images of all nature, or recipes for bad things, or descriptions of crimes, etc.

So if Google is actually in the business of editing the web so that it conforms to puritain standards of appropriateness (which I doubt), it's not very good at it.

> I'm not so sure. Google does deindexing at the request of others. Maybe it also does it on its own and we don't notice it

Google does unilateral deindexing too, and is open that it does so.

Google may temporarily or permanently remove sites from its index and search results if it believes it is obligated to do so by law, if the sites do not meet Google's quality guidelines, or for other reasons, such as if the sites detract from users' ability to locate relevant information. We cannot comment on the individual reasons a page may be removed. However, certain actions such as cloaking, writing text in such a way that it can be seen by search engines but not by users, or setting up pages/links with the sole purpose of fooling search engines may result in removal from our index. Please read our spam policies pages for more information.

https://support.google.com/webmasters/answer/40052?hl=en

People do notice it, and write “what do if it happens to you” articles about it:

https://www.searchenginejournal.com/deindexed-by-google-how-...

The quote is really about spam (although yes, it lets the door open to other removals without having to provide a reason).

Anyway my point is really that current editorial rules of LLMs seem several orders of magnitude more restrictive than those of the Google index, and I think it's interesting to ask why.

>The problem of preventing leaks of forbidden knowledge seems impossible without excluding the forbidden knowledge from the training data. I fail to see how that would be possible without crippling the desirable use cases.

This isnt accurate. There already several tools and methods explicitly for excluding forbidden knowledge from training data, or reweighting the training data so that the LLM have the perspective that the owners want.

e.g. if you want the model from being racist, you run the training data through a classifier to remove data with negative sentiment around race.

https://openai.com/research/dall-e-2-pre-training-mitigation...

that doesn't address the last part "without crippling the desirable use cases." ?

It just cripples the undesirable use cases. It makes a targeted blind spot for the LLM, as decided by their owner.

why is everyone saying the term "forbidden knowledge" like that's just fine and dandy?

because if you dont pretend you want your information censored, you get labeled as an antisocial racist.

How do you define censorship? Is it different from editing and curation?

Is the the choice to exclude some sources from the training data "censorship"? How about the choice of loss minimization strategy?

I think excluding data and sources is both censorship, and something more.

Excluding data is censoring from them model. Then when the model provides incorrect or biased, the firm is actively deceiving the user.

Im not sure what you mean by loss minimization strategy. what I said is focused on the case where data is manipulated with the intent to curate a different perspective than an the source data.

Yes, the fact that this is an extremely hard problem seems to be part of our mythology. Obtaining forbidden knowledge are part of many early creation stories (Garden of Eden - Tree of Knowledge, Prometheus and fire, etc.).

I've thought for decades that everybody would eventually come to their senses and realize that censorship in general is impossible to achieve, but they're still trying as hard as they ever were, and I've come to doubt that they'll ever give up on it.

You have to shift your thinking to realize that their problem isn't with words or even knowledge itself; but with protecting their power or ideology.

In short, fanatics and power mongers are always going to fanatic and power monger.

Everyone has thresholds behind which they store fanaticism. I am sure you would like to see cencored the location and combination to your home safe, your bank balance and passwords, etc. Why? Because distribution of this knowledge might potentially harm you.

This (common) misconstrual of censorship is disingenuous. Scott Alexander proposed a better definition of censorship than "something somebody wants to keep secret": "something that somebody wants to share, and somebody else wants to consume, but a third party wants to suppress".

So if I think you’re a jerk and someone should steal your shit for lolz, so I want to share your address and safe combo, and someone else wants to have them to cash in, but you don’t want that to happen….

Oh they will never give up. Every new generation comes with its horde of fresh censors.

But, the fact that they're trying doesn't mean they will succeed.

And old secrets very often become common knowledge later on.

> I've thought for decades that everybody would eventually come to their senses and realize that censorship in general is impossible to achieve,

Its not, though, and if you think it is, than you don't understand what "censorship" is (a supply-side propaganda mechanism aimed at increasing the average cost of spreading disfavored ideas.)

It's true that complete suppression of ideas (or behaviors) by censorship is generally impossible to achieve, but to mistake that for censorship being impossible to achieve is like saying "making money by work is impossible to achieve" when what you really mean "making enough money to purchase all the property in the world by work is impossible to achieve."

this is the only saving grace in this looming nightmare world of corporate-run omniscient-seeming language-model "oracles" that refuse to espouse any Truth deemed sufficiently politically inconvenient.

Reply on news.ycombinator.com